-- Logs begin at Mon 2022-07-25 11:32:04 UTC, end at Mon 2022-07-25 12:35:22 UTC. -- Jul 25 12:00:27.718854 dv-cinder-797608-61-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jul 25 12:00:27.727373 dv-cinder-797608-61-iscsi devstack@keystone.service[80287]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jul 25 12:00:27.729421 dv-cinder-797608-61-iscsi devstack@keystone.service[80287]: *** Starting uWSGI 2.0.18-debian (64bit) on [Mon Jul 25 12:00:27 2022] *** Jul 25 12:00:27.729421 dv-cinder-797608-61-iscsi devstack@keystone.service[80287]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jul 25 12:00:27.729421 dv-cinder-797608-61-iscsi devstack@keystone.service[80287]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jul 25 12:00:27.729421 dv-cinder-797608-61-iscsi devstack@keystone.service[80287]: nodename: dv-cinder-797608-61-iscsi Jul 25 12:00:27.729421 dv-cinder-797608-61-iscsi devstack@keystone.service[80287]: machine: x86_64 Jul 25 12:00:27.729421 dv-cinder-797608-61-iscsi devstack@keystone.service[80287]: clock source: unix Jul 25 12:00:27.729421 dv-cinder-797608-61-iscsi devstack@keystone.service[80287]: pcre jit disabled Jul 25 12:00:27.729421 dv-cinder-797608-61-iscsi devstack@keystone.service[80287]: detected number of CPU cores: 4 Jul 25 12:00:27.729421 dv-cinder-797608-61-iscsi devstack@keystone.service[80287]: current working directory: / Jul 25 12:00:27.729421 dv-cinder-797608-61-iscsi devstack@keystone.service[80287]: detected binary path: /usr/bin/uwsgi-core Jul 25 12:00:27.729970 dv-cinder-797608-61-iscsi devstack@keystone.service[80287]: your processes number limit is 47680 Jul 25 12:00:27.729970 dv-cinder-797608-61-iscsi devstack@keystone.service[80287]: your memory page size is 4096 bytes Jul 25 12:00:27.729970 dv-cinder-797608-61-iscsi devstack@keystone.service[80287]: detected max file descriptor number: 2048 Jul 25 12:00:27.729970 dv-cinder-797608-61-iscsi devstack@keystone.service[80287]: lock engine: pthread robust mutexes Jul 25 12:00:27.729970 dv-cinder-797608-61-iscsi devstack@keystone.service[80287]: thunder lock: enabled Jul 25 12:00:27.729970 dv-cinder-797608-61-iscsi devstack@keystone.service[80287]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jul 25 12:00:27.729970 dv-cinder-797608-61-iscsi devstack@keystone.service[80287]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Jul 25 12:00:27.769348 dv-cinder-797608-61-iscsi devstack@keystone.service[80287]: Python main interpreter initialized at 0x5606fe5759c0 Jul 25 12:00:27.769348 dv-cinder-797608-61-iscsi devstack@keystone.service[80287]: python threads support enabled Jul 25 12:00:27.769348 dv-cinder-797608-61-iscsi devstack@keystone.service[80287]: your server socket listen backlog is limited to 100 connections Jul 25 12:00:27.769348 dv-cinder-797608-61-iscsi devstack@keystone.service[80287]: your mercy for graceful operations on workers is 90 seconds Jul 25 12:00:27.769724 dv-cinder-797608-61-iscsi devstack@keystone.service[80287]: mapped 403077 bytes (393 KB) for 2 cores Jul 25 12:00:27.769800 dv-cinder-797608-61-iscsi devstack@keystone.service[80287]: *** Operational MODE: preforking *** Jul 25 12:00:27.770039 dv-cinder-797608-61-iscsi devstack@keystone.service[80287]: *** uWSGI is running in multiple interpreter mode *** Jul 25 12:00:27.770039 dv-cinder-797608-61-iscsi devstack@keystone.service[80287]: spawned uWSGI master process (pid: 80287) Jul 25 12:00:27.770248 dv-cinder-797608-61-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jul 25 12:00:27.774296 dv-cinder-797608-61-iscsi devstack@keystone.service[80287]: spawned uWSGI worker 1 (pid: 80297, cores: 1) Jul 25 12:00:27.774296 dv-cinder-797608-61-iscsi devstack@keystone.service[80287]: spawned uWSGI worker 2 (pid: 80298, cores: 1) Jul 25 12:00:27.774296 dv-cinder-797608-61-iscsi devstack@keystone.service[80287]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jul 25 12:00:33.262651 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 25 12:00:33.262651 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: from cryptography.utils import int_from_bytes Jul 25 12:00:33.263442 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 25 12:00:33.263442 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: from cryptography.utils import int_from_bytes Jul 25 12:00:33.271829 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 25 12:00:33.271829 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: from cryptography.utils import int_from_bytes Jul 25 12:00:33.272285 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 25 12:00:33.272285 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: from cryptography.utils import int_from_bytes Jul 25 12:00:33.313218 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 25 12:00:33.313410 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 25 12:00:33.313635 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] command line args: [] {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 25 12:00:33.313912 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 25 12:00:33.314095 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ================================================================================ {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 25 12:00:33.314355 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] admin_token = **** {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 12:00:33.314605 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] config_dir = [] {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 12:00:33.314797 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 12:00:33.314982 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] config_source = [] {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 12:00:33.315165 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] debug = True {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 12:00:33.315368 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 12:00:33.316776 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 12:00:33.316776 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 12:00:33.316776 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 12:00:33.316776 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 12:00:33.316776 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] list_limit = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 12:00:33.317131 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] log_config_append = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 12:00:33.317131 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 12:00:33.317281 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] log_dir = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 12:00:33.317350 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] log_file = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 12:00:33.317549 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 12:00:33.317801 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 12:00:33.318056 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 12:00:33.318308 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 12:00:33.318507 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 12:00:33.318673 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 12:00:33.318921 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 12:00:33.319173 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 12:00:33.319416 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 12:00:33.319646 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 25 12:00:33.319744 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 12:00:33.319870 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 25 12:00:33.320000 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 12:00:33.320106 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] command line args: [] {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 25 12:00:33.320250 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 12:00:33.320355 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 25 12:00:33.320459 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 12:00:33.320598 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] ================================================================================ {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 25 12:00:33.320703 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 12:00:33.320812 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] admin_token = **** {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 12:00:33.320956 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 12:00:33.321062 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] config_dir = [] {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 12:00:33.321164 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 12:00:33.321303 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] config_source = [] {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 12:00:33.321412 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.42/identity {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 12:00:33.321552 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] debug = True {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 12:00:33.321660 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] publish_errors = False {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 12:00:33.321763 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 12:00:33.321925 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 12:00:33.322038 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 12:00:33.322143 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 12:00:33.322247 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 12:00:33.322357 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 12:00:33.322437 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 12:00:33.322508 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 12:00:33.322618 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 12:00:33.322690 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 12:00:33.322758 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] list_limit = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 12:00:33.322825 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] standard_threads = False {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 12:00:33.322957 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] log_config_append = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 12:00:33.323040 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 12:00:33.323118 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 12:00:33.323255 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 12:00:33.323338 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] log_dir = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 12:00:33.323436 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 12:00:33.323518 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] log_file = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 12:00:33.323666 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] use_journal = False {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 12:00:33.323767 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 12:00:33.323849 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] use_json = False {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 12:00:33.323942 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 12:00:33.324018 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] use_stderr = False {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 12:00:33.324153 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] use_syslog = False {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 12:00:33.324233 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 12:00:33.324390 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 12:00:33.324466 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 12:00:33.324612 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 12:00:33.324687 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.324795 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 12:00:33.324882 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.325049 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 12:00:33.325132 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.325303 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 12:00:33.325384 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.325542 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 12:00:33.325618 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.325781 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 12:00:33.325862 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.326020 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 12:00:33.326095 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.326260 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 12:00:33.326344 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.326515 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 12:00:33.326591 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.326769 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 12:00:33.326850 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.327021 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.327103 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 12:00:33.327262 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.327512 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.327660 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.42/identity {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 12:00:33.327776 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] publish_errors = False {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 12:00:33.327859 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.327959 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 12:00:33.328041 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.328199 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 12:00:33.328276 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.328383 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 12:00:33.328482 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.328637 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 12:00:33.328743 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.328878 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 12:00:33.329003 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.329634 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] standard_threads = False {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 12:00:33.329773 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.329874 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 12:00:33.330017 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.330125 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 12:00:33.330277 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.330360 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 12:00:33.330474 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] use_journal = False {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 12:00:33.330554 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.330656 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] use_json = False {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 12:00:33.330789 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.330871 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] use_stderr = False {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 12:00:33.331003 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] use_syslog = False {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 12:00:33.331083 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.331241 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 12:00:33.331323 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.331488 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.331581 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.331750 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.331836 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.331991 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.332067 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.332244 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.332328 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.332498 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.332573 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.332734 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.332808 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.332969 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.333045 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.333215 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.333301 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.333466 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.333546 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.333708 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.333786 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.333956 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.334030 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.334190 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.334264 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.334432 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.334509 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.334673 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.334747 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.334905 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.334979 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.335142 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.335218 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.335377 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.335697 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.335784 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.335855 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.335932 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.336002 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.336115 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.336192 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.336353 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.336440 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.336596 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.336672 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.336832 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.336907 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.337074 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.337161 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.337315 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.337392 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.337569 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.337645 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.337805 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.337960 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.338039 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.338110 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.338274 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.338363 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.338515 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.338602 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.338756 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.338831 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.338994 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.339067 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.339226 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.339308 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.339618 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.339750 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.339750 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.339932 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.340038 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.340145 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.340247 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.340340 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.340440 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.340555 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.340666 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.340789 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.340893 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.341008 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.341120 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.341231 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.341364 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.341488 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.341602 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.341713 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.341824 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.341937 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.342054 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.342174 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.342294 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.342407 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.342509 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.342618 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.342729 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.342848 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.342970 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.343083 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.343194 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.343312 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.343423 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.343525 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.343701 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.343817 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.343929 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.344032 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.344142 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.344253 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.344372 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.344486 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.344597 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.344707 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.345420 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.345541 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.345663 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.345777 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.345889 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.346000 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.346112 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.346232 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.346353 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.346467 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.346585 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.346697 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.346809 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.346929 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.347050 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.347165 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.347277 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.347389 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.347500 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.347639 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.347765 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.347880 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.347998 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.348111 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.348224 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.348336 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.348459 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.348573 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.348685 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.348803 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.348922 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.349034 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.349153 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.349269 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.349381 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.349493 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.349604 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.349717 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.349903 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.350020 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.350139 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.350253 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.350365 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.350477 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.350597 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] auth.external = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.350719 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.350844 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.350958 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.351071 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.351184 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.351297 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.351409 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.351532 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] auth.password = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.351663 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.351786 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] auth.token = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.351900 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.352018 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.352139 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.352253 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.352366 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.352478 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.352590 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.352701 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.352821 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.352935 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.353047 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.353159 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.353271 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.353384 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.353504 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.353617 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] credential.caching = True {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.353735 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.353848 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.353961 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.354072 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.354218 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.354341 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.354466 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.354586 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.354701 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.354813 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.354934 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.355052 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.355177 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.355292 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.355405 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.355519 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.355646 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.355760 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.355889 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.356004 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.356122 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.356236 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.356349 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.356461 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.356582 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.356697 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.356810 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] auth.external = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.356922 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.357035 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.357148 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.357268 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.357384 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] federation.caching = True {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.357497 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.357610 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.357723 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] auth.password = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.357836 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.357957 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] auth.token = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.358072 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.358185 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.358297 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.358410 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.358523 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.358646 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.358764 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.358888 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.359003 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.359117 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.359231 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.359352 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.359467 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.359660 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.359779 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.359892 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] credential.caching = True {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.360005 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.360126 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.360241 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] identity.caching = True {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.360354 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.360467 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.360581 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.360693 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.360816 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.360931 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.361044 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.361157 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.361269 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.361382 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.361503 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.361618 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.361731 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.361845 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.361958 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.362071 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.362192 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.362307 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.362420 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.362533 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.362645 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.362758 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.362880 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.362995 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.363118 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.363232 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.363345 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.363458 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.363661 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] federation.caching = True {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.363786 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.363903 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.364016 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.364130 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.364251 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.364366 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.364479 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.364592 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.364705 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.364818 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.364940 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.365055 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.365168 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.365281 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.365395 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.365508 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.365629 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.365744 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.365856 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.365971 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.366084 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.366196 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.366318 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.366432 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.366547 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] identity.caching = True {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.366659 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.366780 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.366895 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.367017 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.367132 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.367245 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.367358 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.367471 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.367595 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.367722 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.367838 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.367951 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.368064 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.368178 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.368291 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.368412 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.368530 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.368644 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.368757 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.368870 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.368989 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.369112 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.369227 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.369340 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.369458 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.369650 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.369768 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.369892 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.370007 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.370142 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.370260 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.370396 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.370514 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.370636 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.370750 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.370863 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.370976 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.371088 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.371202 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.371323 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.371438 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.371597 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.371715 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.371828 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.371941 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.372069 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.372185 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.372306 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.user = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.372420 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.372621 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.372748 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.372876 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.372992 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.373105 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.373218 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.373331 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.373444 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.373564 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.373679 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.373792 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.373905 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.374017 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.374130 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.374254 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.374370 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.374484 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.374596 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.374709 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.374830 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.374950 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.375070 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.375184 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.375297 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.375411 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.375531 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.375531 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.375765 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.375878 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.375990 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.376103 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.376228 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.376344 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.376458 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.376571 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.376684 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.376796 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.376918 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.377033 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.377146 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.377259 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.377371 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.377484 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.377606 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.377721 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.377834 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.377947 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.378059 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.378172 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.378293 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.378408 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] ldap.user = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.378526 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.378645 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.378760 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.378873 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.378994 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.379109 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.379222 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.379335 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.379448 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.379606 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.379735 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.379851 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.379965 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.380077 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.380189 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.380302 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.380422 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.380537 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.380651 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.380764 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.380877 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.380989 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.381109 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] resource.caching = True {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.381223 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.381337 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.381450 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.381563 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.381675 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.381796 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.381911 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.382023 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.382136 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.382248 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.382360 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.382481 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.382595 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.382708 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.382821 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.382933 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.383046 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.383166 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.383281 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] role.caching = True {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.383393 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.383599 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] role.driver = sql {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.383720 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.383836 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.383961 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.384078 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.384197 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.384311 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.384425 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.384538 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.384659 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.384774 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.384888 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.385000 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.385113 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.385226 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.385347 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.385462 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.385575 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.385688 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.385800 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.385914 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.386096 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.386212 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.386326 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.386439 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.386552 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.386665 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.386787 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.386901 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.387015 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.387128 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] resource.caching = True {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.387239 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.387361 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.387476 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.387642 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.387758 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.387882 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.387998 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.388119 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.388234 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.388350 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.388464 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.388578 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.388690 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.388811 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.388926 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.389040 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.389159 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.389273 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.389386 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.389506 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] role.caching = True {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.389621 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.389734 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] role.driver = sql {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.389846 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.389959 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.390073 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.390193 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.390308 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.390421 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.390534 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.390647 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.390759 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.390880 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.390995 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.391108 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.391223 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.391335 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.391449 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.392111 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.392234 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] token.caching = True {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.392348 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.392461 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.392595 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.392713 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.392834 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.392950 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.393063 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.393177 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.393317 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.393434 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.393556 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.393671 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.393784 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.393944 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.394061 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.394175 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.394297 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.394412 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.394525 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.394666 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.394783 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.394920 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.395046 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.395160 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.395272 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.395386 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.395498 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.395674 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.395790 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.395903 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.396017 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.396129 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.396241 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.396362 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.396478 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] database.connection = **** {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.396600 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.396722 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.396838 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.396951 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.397064 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.397176 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.397316 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.397431 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.397545 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.397657 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.397769 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.397881 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.398002 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.398116 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.398229 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.398345 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.398458 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] token.caching = True {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.398571 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.398692 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.398806 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.398919 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.399031 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.399143 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.399255 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.399381 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.399496 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.399625 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.399749 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.399864 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.399977 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.400098 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.400213 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.400326 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.400439 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.400551 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.400665 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.400786 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 25 12:00:33.400901 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.401013 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.401236 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.401487 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.401740 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.401974 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.402223 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.402520 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.402788 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] database.connection = **** {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.403037 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.403293 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.403534 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.403804 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.404044 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.404286 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.404528 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.404769 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.405018 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.405260 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.405507 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.405748 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.406006 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.406247 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.406507 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.406746 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.406995 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.407288 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 12:00:33.407982 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 25 12:00:33.430438 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80297) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 12:00:33.430675 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80297) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 12:00:33.430906 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80297) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 12:00:33.432726 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80298) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 12:00:33.432968 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80298) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 12:00:33.433202 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80298) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 12:00:33.439843 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80297) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 12:00:33.442092 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80298) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 12:00:33.469758 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80297) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 12:00:33.469997 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80297) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 12:00:33.470159 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80298) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 12:00:33.470400 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80298) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 12:00:33.525441 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80297) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 12:00:33.531068 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80298) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 12:00:33.577202 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80297) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 12:00:33.577444 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80297) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 12:00:33.578135 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80298) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 12:00:33.578381 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80298) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 12:00:33.642650 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80297) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 12:00:33.642876 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80297) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 12:00:33.643104 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80297) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 12:00:33.643188 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80298) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 12:00:33.643327 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80297) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 12:00:33.643411 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80298) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 12:00:33.643563 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80297) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 12:00:33.643646 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80298) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 12:00:33.643792 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80297) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 12:00:33.643875 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80298) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 12:00:33.644070 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80298) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 12:00:33.644295 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80298) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 12:00:33.666988 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80297) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 12:00:33.667233 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80297) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 12:00:33.667327 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80298) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 12:00:33.667484 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80297) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 12:00:33.667590 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80298) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 12:00:33.667748 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80297) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 12:00:33.667831 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80298) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 12:00:33.667976 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80297) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 12:00:33.668072 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80298) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 12:00:33.668204 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80297) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 12:00:33.668286 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80298) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 12:00:33.668422 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80297) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 12:00:33.668503 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80298) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 12:00:33.668660 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80298) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 12:00:33.679847 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80298) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 12:00:33.679945 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80297) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 12:00:33.693117 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80298) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 12:00:33.693303 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80297) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 12:00:33.693388 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80298) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 12:00:33.693541 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80297) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 12:00:33.693624 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80298) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 12:00:33.693773 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80297) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 12:00:33.693861 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80298) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 12:00:33.694002 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80297) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 12:00:33.694092 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80298) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 12:00:33.694223 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80298) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 12:00:33.694304 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80297) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 12:00:33.694440 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80297) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 12:00:33.694520 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80298) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 12:00:33.694673 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80297) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 12:00:33.712411 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:00:33.712627 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 25 12:00:33.712972 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:00:33.713191 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:00:33.713276 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 25 12:00:33.713391 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 25 12:00:33.713543 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:00:33.713692 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:00:33.713774 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:00:33.713913 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 25 12:00:33.714004 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:00:33.714182 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:00:33.714263 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:00:33.714441 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:00:33.714697 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:00:33.714951 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:00:33.725119 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:00:33.725401 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:00:33.725677 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:00:33.725759 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:00:33.725967 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:00:33.726062 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:00:33.726274 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:00:33.726559 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:00:33.734281 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80298) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 12:00:33.734770 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80297) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 12:00:33.737620 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80298) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 12:00:33.737907 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:00:33.738089 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80297) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 12:00:33.738197 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:00:33.738373 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:00:33.738484 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:00:33.738658 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:00:33.738773 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:00:33.738943 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:00:33.739024 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:00:33.739230 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:00:33.739319 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:00:33.739493 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:00:33.739591 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:00:33.739786 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:00:33.739867 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:00:33.740060 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:00:33.740153 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:00:33.740331 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:00:33.740415 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:00:33.740605 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:00:33.740875 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:00:33.769329 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:00:33.769743 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:00:33.770947 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80298) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 12:00:33.771234 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:00:33.771365 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80297) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 12:00:33.771668 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:00:33.777486 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80298) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 12:00:33.777486 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:00:33.777848 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80297) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 12:00:33.777956 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:00:33.778073 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:00:33.778073 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:00:33.784797 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80298) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 12:00:33.785086 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:00:33.785396 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80297) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 12:00:33.785682 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:00:33.789831 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80298) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 12:00:33.790120 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:00:33.790406 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:00:33.790654 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80297) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 12:00:33.790751 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:00:33.790945 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:00:33.791026 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:00:33.791220 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:00:33.791300 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:00:33.791496 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:00:33.791612 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:00:33.791794 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:00:33.791877 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:00:33.792064 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:00:33.792334 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:00:33.792601 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:00:33.811687 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:00:33.811792 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:00:33.811968 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:00:33.812051 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:00:33.817088 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80297) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 12:00:33.817181 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80298) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 12:00:33.820761 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:00:33.820946 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:00:33.821159 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:00:33.821253 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:00:33.826129 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80297) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 12:00:33.826316 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80298) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 12:00:33.829794 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80297) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 12:00:33.830005 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80298) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 12:00:33.833445 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:00:33.833654 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:00:33.833737 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:00:33.833958 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:00:33.834039 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:00:33.834247 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:00:33.834340 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:00:33.834537 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:00:33.834618 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:00:33.834820 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:00:33.834901 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:00:33.835115 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:00:33.853736 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80297) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 12:00:33.854023 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:00:33.854156 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80298) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 12:00:33.854293 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:00:33.854447 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:00:33.854558 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:00:33.854722 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:00:33.854991 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:00:33.862973 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:00:33.863465 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:00:33.864695 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:00:33.864975 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:00:33.865318 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:00:33.865600 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:00:33.868170 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80297) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 12:00:33.868456 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:00:33.868684 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80298) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 12:00:33.868771 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:00:33.868977 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:00:33.869067 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:00:33.869263 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:00:33.869345 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:00:33.869548 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:00:33.869834 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:00:33.883389 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80297) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 12:00:33.883701 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:00:33.883786 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80298) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 12:00:33.883984 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:00:33.884068 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:00:33.884277 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:00:33.884375 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:00:33.884626 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:00:33.884708 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:00:33.884913 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:00:33.884992 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:00:33.885187 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:00:33.885268 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:00:33.885477 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:00:33.903354 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80297) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 12:00:33.903818 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80298) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 12:00:33.906621 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80297) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 12:00:33.907074 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80298) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 12:00:33.910525 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:00:33.911092 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:00:33.912304 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:00:33.912740 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:00:33.914211 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80297) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 12:00:33.914493 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:00:33.914658 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80298) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 12:00:33.914775 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:00:33.914944 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:00:33.915231 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:00:33.922009 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:00:33.922515 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:00:33.923826 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80297) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 12:00:33.924189 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80298) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 12:00:33.927029 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:00:33.927319 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:00:33.927663 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:00:33.927746 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:00:33.927956 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:00:33.928038 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:00:33.928243 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:00:33.928534 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:00:33.937033 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80297) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 12:00:33.937325 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:00:33.937609 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:00:33.937702 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80298) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 12:00:33.937907 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:00:33.938195 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:00:33.945432 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80297) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 12:00:33.945712 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:00:33.945990 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:00:33.946185 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80298) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 12:00:33.946266 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:00:33.946471 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:00:33.946551 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:00:33.946759 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:00:33.946893 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:00:33.947045 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:00:33.947172 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:00:33.947335 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:00:33.947453 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:00:33.947636 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:00:33.947755 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:00:33.947917 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:00:33.948039 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:00:33.948203 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:00:33.948323 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:00:33.948485 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:00:33.948587 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:00:33.948765 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:00:33.948854 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:00:33.949059 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:00:33.949140 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:00:33.949325 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:00:33.949609 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:00:33.949875 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 12:00:33.988754 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x5606fe5759c0 pid: 80297 (default app) Jul 25 12:00:33.992176 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db2ecac-6d8e-40f3-9da2-b64397960f29 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:00:33.992478 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db2ecac-6d8e-40f3-9da2-b64397960f29 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:00:33.992764 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db2ecac-6d8e-40f3-9da2-b64397960f29 None None] PATH_INFO: `/v3` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:00:33.994067 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x5606fe5759c0 pid: 80298 (default app) Jul 25 12:00:33.996974 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1/1] 10.222.0.42 () {50 vars in 791 bytes} [Mon Jul 25 12:00:33 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:00:41.656416 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d4fc0b-2513-4bbb-adf6-027a45ecfca0 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:00:41.656982 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d4fc0b-2513-4bbb-adf6-027a45ecfca0 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:00:41.656982 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d4fc0b-2513-4bbb-adf6-027a45ecfca0 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:00:41.661940 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1/2] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 25 12:00:41 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 12:00:41.667332 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad97c7a-115e-4701-a57e-901da18e8fb3 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:00:41.667797 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad97c7a-115e-4701-a57e-901da18e8fb3 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:00:41.667929 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad97c7a-115e-4701-a57e-901da18e8fb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:00:41.738579 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: INFO dbcounter [None req-fad97c7a-115e-4701-a57e-901da18e8fb3 None None] Registered counter for database keystone Jul 25 12:00:41.760312 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG oslo_db.sqlalchemy.engines [None req-fad97c7a-115e-4701-a57e-901da18e8fb3 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80297) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 25 12:00:41.766478 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writer thread running {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 25 12:00:41.946752 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG passlib.handlers.bcrypt [None req-fad97c7a-115e-4701-a57e-901da18e8fb3 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80297) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 25 12:00:41.947097 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG passlib.handlers.bcrypt [None req-fad97c7a-115e-4701-a57e-901da18e8fb3 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80297) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 25 12:00:42.000720 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-fad97c7a-115e-4701-a57e-901da18e8fb3 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:00:42.002872 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-fad97c7a-115e-4701-a57e-901da18e8fb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:00:42.127785 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 2/3] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 25 12:00:41 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 461 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jul 25 12:00:42.293387 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d4fc0b-2513-4bbb-adf6-027a45ecfca0 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:00:42.369576 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: INFO dbcounter [None req-a4d4fc0b-2513-4bbb-adf6-027a45ecfca0 None None] Registered counter for database keystone Jul 25 12:00:42.389481 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG oslo_db.sqlalchemy.engines [None req-a4d4fc0b-2513-4bbb-adf6-027a45ecfca0 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80298) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 25 12:00:42.395795 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writer thread running {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 25 12:00:42.514786 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d4fc0b-2513-4bbb-adf6-027a45ecfca0 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:00:42.519127 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e9977f-62a2-41bb-86ff-76dcdb476447 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:00:42.519818 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e9977f-62a2-41bb-86ff-76dcdb476447 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:00:42.520112 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e9977f-62a2-41bb-86ff-76dcdb476447 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:00:42.520399 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e9977f-62a2-41bb-86ff-76dcdb476447 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:00:42.528569 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e9977f-62a2-41bb-86ff-76dcdb476447 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:00:42.578963 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: WARNING keystone.common.rbac_enforcer.enforcer [None req-f3e9977f-62a2-41bb-86ff-76dcdb476447 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 25 12:00:42.579261 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e9977f-62a2-41bb-86ff-76dcdb476447 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:00:42.589318 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 2/4] 10.222.0.42 () {58 vars in 1324 bytes} [Mon Jul 25 12:00:42 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 25 12:00:42.595361 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad97c7a-115e-4701-a57e-901da18e8fb3 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:00:42.609604 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad97c7a-115e-4701-a57e-901da18e8fb3 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:00:42.614013 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a9d8aa-6c84-4105-ae38-b52b5ffa86df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:00:42.615126 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a9d8aa-6c84-4105-ae38-b52b5ffa86df admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:00:42.615418 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a9d8aa-6c84-4105-ae38-b52b5ffa86df admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:00:42.615730 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a9d8aa-6c84-4105-ae38-b52b5ffa86df admin admin] PATH_INFO: `/v3/projects/827585a7cfe8411987f9b81102c1c103` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:00:42.617215 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a9d8aa-6c84-4105-ae38-b52b5ffa86df admin admin] RBAC: Authorizing `identity:get_project(project_id=827585a7cfe8411987f9b81102c1c103)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:00:42.669684 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: WARNING keystone.common.rbac_enforcer.enforcer [None req-38a9d8aa-6c84-4105-ae38-b52b5ffa86df admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 25 12:00:42.670222 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a9d8aa-6c84-4105-ae38-b52b5ffa86df admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:00:42.675114 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 3/5] 10.222.0.42 () {56 vars in 1207 bytes} [Mon Jul 25 12:00:42 2022] GET /identity/v3/projects/827585a7cfe8411987f9b81102c1c103 => generated 344 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:00:44.634393 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e127e312-4bde-46c9-b71c-5919c3991850 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:00:44.634931 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e127e312-4bde-46c9-b71c-5919c3991850 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:00:44.634931 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e127e312-4bde-46c9-b71c-5919c3991850 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:00:44.639092 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 3/6] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 25 12:00:44 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:00:44.644966 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7441d96c-4793-44a2-a9a6-14b71ebe2e11 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:00:44.645392 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7441d96c-4793-44a2-a9a6-14b71ebe2e11 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:00:44.645551 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7441d96c-4793-44a2-a9a6-14b71ebe2e11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:00:44.666210 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-7441d96c-4793-44a2-a9a6-14b71ebe2e11 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:00:44.667777 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-7441d96c-4793-44a2-a9a6-14b71ebe2e11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:00:44.675689 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 4/7] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 25 12:00:44 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jul 25 12:00:44.697495 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e127e312-4bde-46c9-b71c-5919c3991850 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:00:44.710283 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e127e312-4bde-46c9-b71c-5919c3991850 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:00:44.714541 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55626885-4b2d-4d27-bbde-8195031286dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:00:44.715178 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55626885-4b2d-4d27-bbde-8195031286dc admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:00:44.715472 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55626885-4b2d-4d27-bbde-8195031286dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:00:44.715792 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55626885-4b2d-4d27-bbde-8195031286dc admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:00:44.723552 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55626885-4b2d-4d27-bbde-8195031286dc admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:00:44.725341 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55626885-4b2d-4d27-bbde-8195031286dc admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:00:44.734137 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 4/8] 10.222.0.42 () {58 vars in 1324 bytes} [Mon Jul 25 12:00:44 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 25 12:00:44.739161 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7441d96c-4793-44a2-a9a6-14b71ebe2e11 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:00:44.748134 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7441d96c-4793-44a2-a9a6-14b71ebe2e11 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:00:44.753023 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adfde9a-48f0-4c09-b355-595be5c09dea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:00:44.753877 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adfde9a-48f0-4c09-b355-595be5c09dea admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:00:44.754171 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adfde9a-48f0-4c09-b355-595be5c09dea admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:00:44.754458 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adfde9a-48f0-4c09-b355-595be5c09dea admin admin] PATH_INFO: `/v3/users/0fad2691baf0472d99015f9f123f8d32` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:00:44.755894 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1adfde9a-48f0-4c09-b355-595be5c09dea admin admin] RBAC: Authorizing `identity:get_user(user_id=0fad2691baf0472d99015f9f123f8d32)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:00:44.757715 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1adfde9a-48f0-4c09-b355-595be5c09dea admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:00:44.759266 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 5/9] 10.222.0.42 () {56 vars in 1198 bytes} [Mon Jul 25 12:00:44 2022] GET /identity/v3/users/0fad2691baf0472d99015f9f123f8d32 => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:00:46.779310 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f125978-be3c-47dd-b26f-1f8a1eab6ae0 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:00:46.779310 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f125978-be3c-47dd-b26f-1f8a1eab6ae0 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:00:46.779865 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f125978-be3c-47dd-b26f-1f8a1eab6ae0 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:00:46.780931 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 5/10] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 25 12:00:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 12:00:46.786160 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a37021-d1ec-45ff-ac13-c4661231119c None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:00:46.786563 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a37021-d1ec-45ff-ac13-c4661231119c None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:00:46.786790 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a37021-d1ec-45ff-ac13-c4661231119c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:00:46.805087 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-e4a37021-d1ec-45ff-ac13-c4661231119c None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:00:46.806251 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-e4a37021-d1ec-45ff-ac13-c4661231119c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:00:46.841933 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82cdbef-c168-423f-8c5f-85c30b4aaaa6 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:00:46.842228 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82cdbef-c168-423f-8c5f-85c30b4aaaa6 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:00:46.842519 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82cdbef-c168-423f-8c5f-85c30b4aaaa6 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:00:46.843902 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 6/11] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 25 12:00:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 12:00:46.848909 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef95514-410a-447a-bbb2-d06dfd8ee403 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:00:46.849205 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef95514-410a-447a-bbb2-d06dfd8ee403 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:00:46.849489 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef95514-410a-447a-bbb2-d06dfd8ee403 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:00:46.886511 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 6/12] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 25 12:00:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 12:00:46.892663 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG passlib.handlers.bcrypt [None req-fef95514-410a-447a-bbb2-d06dfd8ee403 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80298) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 25 12:00:46.893084 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG passlib.handlers.bcrypt [None req-fef95514-410a-447a-bbb2-d06dfd8ee403 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80298) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 25 12:00:46.909029 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a37021-d1ec-45ff-ac13-c4661231119c None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:00:46.932249 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:00:46.938624 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-fef95514-410a-447a-bbb2-d06dfd8ee403 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:00:46.940768 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-fef95514-410a-447a-bbb2-d06dfd8ee403 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:00:47.012048 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a37021-d1ec-45ff-ac13-c4661231119c None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:00:47.039521 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 7/13] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 25 12:00:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 25 12:00:47.045634 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c4c557-6e1c-45f0-b796-a1092f484683 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:00:47.046437 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c4c557-6e1c-45f0-b796-a1092f484683 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:00:47.046732 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c4c557-6e1c-45f0-b796-a1092f484683 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:00:47.047020 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c4c557-6e1c-45f0-b796-a1092f484683 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:00:47.058704 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c4c557-6e1c-45f0-b796-a1092f484683 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 12:00:47.061336 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef95514-410a-447a-bbb2-d06dfd8ee403 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:00:47.069927 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-a1c4c557-6e1c-45f0-b796-a1092f484683 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 12:00:47.069927 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:00:47.069927 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:00:47.069927 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:00:47.069927 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:00:47.069927 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:00:47.069927 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:00:47.069927 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:00:47.069927 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:00:47.069927 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:00:47.069927 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:00:47.069927 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:00:47.069927 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 12:00:47.069927 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 12:00:47.069927 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 12:00:47.069927 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 12:00:47.070825 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 12:00:47.070825 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 12:00:47.070825 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 12:00:47.070825 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 12:00:47.070825 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:00:47.070825 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:00:47.070825 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:00:47.070825 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:00:47.070825 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:00:47.070825 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:00:47.070825 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:00:47.070825 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 25 12:00:47.070825 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:00:47.070825 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:00:47.070825 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:00:47.070825 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:00:47.071693 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:00:47.071693 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:00:47.071693 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:00:47.071693 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:00:47.071693 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 12:00:47.071693 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 12:00:47.071693 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 12:00:47.071693 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 12:00:47.071693 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 12:00:47.071693 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 12:00:47.071693 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 12:00:47.071693 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 25 12:00:47.071693 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 7/14] 10.222.0.42 () {56 vars in 1096 bytes} [Mon Jul 25 12:00:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 12:00:47.076776 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c4c557-6e1c-45f0-b796-a1092f484683 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:00:47.160204 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:00:47.162929 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef95514-410a-447a-bbb2-d06dfd8ee403 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:00:47.165821 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:00:47.180057 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c4c557-6e1c-45f0-b796-a1092f484683 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:00:47.204051 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f98df1c-e8a7-40cf-935c-6a05456347c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:00:47.204951 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f98df1c-e8a7-40cf-935c-6a05456347c1 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:00:47.205253 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f98df1c-e8a7-40cf-935c-6a05456347c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:00:47.205543 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f98df1c-e8a7-40cf-935c-6a05456347c1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:00:47.214117 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041f2c5e-519b-49db-9e85-e0d8dcbb0297 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:00:47.214788 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041f2c5e-519b-49db-9e85-e0d8dcbb0297 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:00:47.215081 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041f2c5e-519b-49db-9e85-e0d8dcbb0297 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:00:47.215367 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041f2c5e-519b-49db-9e85-e0d8dcbb0297 None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:00:47.216173 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041f2c5e-519b-49db-9e85-e0d8dcbb0297 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:00:47.216489 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041f2c5e-519b-49db-9e85-e0d8dcbb0297 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:00:47.217905 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041f2c5e-519b-49db-9e85-e0d8dcbb0297 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:00:47.230207 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 8/15] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Jul 25 12:00:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 12:00:47.236264 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041f2c5e-519b-49db-9e85-e0d8dcbb0297 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:00:47.274334 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f98df1c-e8a7-40cf-935c-6a05456347c1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:00:47.275891 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f98df1c-e8a7-40cf-935c-6a05456347c1 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:00:47.333738 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041f2c5e-519b-49db-9e85-e0d8dcbb0297 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:00:47.366992 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c8ff52-5fe1-4bde-8df3-1fa29ac6c5e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:00:47.367894 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c8ff52-5fe1-4bde-8df3-1fa29ac6c5e1 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:00:47.368193 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c8ff52-5fe1-4bde-8df3-1fa29ac6c5e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:00:47.368481 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c8ff52-5fe1-4bde-8df3-1fa29ac6c5e1 None admin] PATH_INFO: `/v3/users/0fad2691baf0472d99015f9f123f8d32` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:00:47.369704 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c8ff52-5fe1-4bde-8df3-1fa29ac6c5e1 None admin] RBAC: Authorizing `identity:get_user(user_id=0fad2691baf0472d99015f9f123f8d32)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:00:47.371157 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c8ff52-5fe1-4bde-8df3-1fa29ac6c5e1 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:00:47.372572 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 8/16] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Jul 25 12:00:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 25 12:00:47.377779 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 9/17] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Jul 25 12:00:47 2022] GET /identity/v3/users/0fad2691baf0472d99015f9f123f8d32 => generated 247 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 12:00:47.378678 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f98df1c-e8a7-40cf-935c-6a05456347c1 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:00:47.378933 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c8ff52-5fe1-4bde-8df3-1fa29ac6c5e1 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:00:47.382055 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:00:47.393688 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:00:47.476398 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c8ff52-5fe1-4bde-8df3-1fa29ac6c5e1 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:00:47.477199 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f98df1c-e8a7-40cf-935c-6a05456347c1 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:00:47.510821 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2b5788-c656-48aa-a38c-757f57a1a619 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:00:47.511653 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2b5788-c656-48aa-a38c-757f57a1a619 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:00:47.511948 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2b5788-c656-48aa-a38c-757f57a1a619 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:00:47.512233 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2b5788-c656-48aa-a38c-757f57a1a619 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:00:47.516150 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e53d633-8bdf-4f36-9a06-4abef0cb3d5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:00:47.517233 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e53d633-8bdf-4f36-9a06-4abef0cb3d5e None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:00:47.517532 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e53d633-8bdf-4f36-9a06-4abef0cb3d5e None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:00:47.517824 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e53d633-8bdf-4f36-9a06-4abef0cb3d5e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:00:47.519218 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e53d633-8bdf-4f36-9a06-4abef0cb3d5e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:00:47.520753 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e53d633-8bdf-4f36-9a06-4abef0cb3d5e None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:00:47.522344 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2b5788-c656-48aa-a38c-757f57a1a619 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:00:47.522442 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 9/18] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jul 25 12:00:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:00:47.523838 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2b5788-c656-48aa-a38c-757f57a1a619 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:00:47.528090 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e53d633-8bdf-4f36-9a06-4abef0cb3d5e None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:00:47.542549 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-ba2b5788-c656-48aa-a38c-757f57a1a619 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:00:47.542549 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:00:47.542549 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:00:47.542549 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:00:47.542549 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:00:47.542549 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:00:47.542549 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:00:47.542549 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:00:47.542549 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:00:47.542549 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:00:47.542549 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:00:47.542549 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:00:47.542549 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 12:00:47.542549 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 12:00:47.542549 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 12:00:47.543389 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 12:00:47.543389 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:00:47.543389 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:00:47.543389 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:00:47.543389 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:00:47.543389 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:00:47.543389 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:00:47.543389 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:00:47.543389 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 25 12:00:47.543389 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:00:47.543389 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:00:47.543389 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:00:47.543389 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:00:47.543389 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:00:47.543389 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:00:47.543389 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:00:47.543389 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:00:47.544218 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 12:00:47.544218 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 12:00:47.544218 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:00:47.544218 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 25 12:00:47.544218 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 10/19] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jul 25 12:00:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 12:00:47.550145 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2b5788-c656-48aa-a38c-757f57a1a619 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:00:47.601234 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:00:47.612936 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:00:47.631710 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e53d633-8bdf-4f36-9a06-4abef0cb3d5e None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:00:47.652901 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2b5788-c656-48aa-a38c-757f57a1a619 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:00:47.664690 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f57d9b-c261-44b4-a6ad-9c9562d0f4de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:00:47.665349 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f57d9b-c261-44b4-a6ad-9c9562d0f4de None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:00:47.665644 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f57d9b-c261-44b4-a6ad-9c9562d0f4de None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:00:47.665926 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f57d9b-c261-44b4-a6ad-9c9562d0f4de None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:00:47.666765 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f57d9b-c261-44b4-a6ad-9c9562d0f4de None admin] RBAC: Adding query filter params (role.id=a7d20885b6d94928a34b3cc5b3a00006, scope.domain.id=default, user.id=0fad2691baf0472d99015f9f123f8d32) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:00:47.667082 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f57d9b-c261-44b4-a6ad-9c9562d0f4de None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:00:47.668581 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f57d9b-c261-44b4-a6ad-9c9562d0f4de None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:00:47.677937 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 10/20] 10.222.0.42 () {56 vars in 1322 bytes} [Mon Jul 25 12:00:47 2022] GET /identity/v3/role_assignments?user.id=0fad2691baf0472d99015f9f123f8d32&scope.domain.id=default&role.id=a7d20885b6d94928a34b3cc5b3a00006 => generated 233 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 12:00:47.686568 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad09f7c-f091-4714-ac78-7550e337d82e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:00:47.687262 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad09f7c-f091-4714-ac78-7550e337d82e None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:00:47.687567 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad09f7c-f091-4714-ac78-7550e337d82e None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:00:47.687879 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad09f7c-f091-4714-ac78-7550e337d82e None admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:00:47.688631 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad09f7c-f091-4714-ac78-7550e337d82e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:00:47.689510 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad09f7c-f091-4714-ac78-7550e337d82e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:00:47.690955 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad09f7c-f091-4714-ac78-7550e337d82e None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:00:47.710665 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 11/21] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jul 25 12:00:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 12:00:50.110492 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3b2d47-2974-412c-86b0-9ab166d5beca None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:00:50.111573 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3b2d47-2974-412c-86b0-9ab166d5beca None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:00:50.112075 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3b2d47-2974-412c-86b0-9ab166d5beca None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:00:50.113638 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 11/22] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 25 12:00:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:00:50.119044 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e68629-94fd-467f-bbe4-897f55e44ed7 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:00:50.119456 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e68629-94fd-467f-bbe4-897f55e44ed7 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:00:50.119637 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e68629-94fd-467f-bbe4-897f55e44ed7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:00:50.149622 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-d1e68629-94fd-467f-bbe4-897f55e44ed7 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:00:50.150918 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-d1e68629-94fd-467f-bbe4-897f55e44ed7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:00:50.253148 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 12/23] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 25 12:00:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 12:00:50.288113 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3b2d47-2974-412c-86b0-9ab166d5beca None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:00:50.291415 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ef3be7-4f55-48c7-bba9-2b1ad9ee245e None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:00:50.291743 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ef3be7-4f55-48c7-bba9-2b1ad9ee245e None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:00:50.292031 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ef3be7-4f55-48c7-bba9-2b1ad9ee245e None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:00:50.293842 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 13/24] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 25 12:00:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 12:00:50.299324 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e52a11-29e7-46e0-86f7-300ad1f89370 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:00:50.299655 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e52a11-29e7-46e0-86f7-300ad1f89370 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:00:50.299940 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e52a11-29e7-46e0-86f7-300ad1f89370 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:00:50.447617 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-32e52a11-29e7-46e0-86f7-300ad1f89370 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:00:50.448791 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-32e52a11-29e7-46e0-86f7-300ad1f89370 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:00:50.514317 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3b2d47-2974-412c-86b0-9ab166d5beca None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:00:50.555899 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:00:50.585429 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f206a95a-d4fc-4e0d-a6bf-adf25f47a5e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:00:50.586239 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f206a95a-d4fc-4e0d-a6bf-adf25f47a5e7 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:00:50.586531 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f206a95a-d4fc-4e0d-a6bf-adf25f47a5e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:00:50.586823 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f206a95a-d4fc-4e0d-a6bf-adf25f47a5e7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:00:50.589621 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 14/25] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 25 12:00:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 291 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 25 12:00:50.590686 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdc8eec-4cdb-4f54-bd47-f00874e95af5 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:00:50.590983 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdc8eec-4cdb-4f54-bd47-f00874e95af5 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:00:50.591269 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdc8eec-4cdb-4f54-bd47-f00874e95af5 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:00:50.594099 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 15/26] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 25 12:00:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:00:50.602849 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d084560e-597f-48a9-a2d4-65b70d443d96 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:00:50.603145 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d084560e-597f-48a9-a2d4-65b70d443d96 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:00:50.603434 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d084560e-597f-48a9-a2d4-65b70d443d96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:00:50.606444 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:00:50.642936 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-d084560e-597f-48a9-a2d4-65b70d443d96 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:00:50.644180 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-d084560e-597f-48a9-a2d4-65b70d443d96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:00:50.647149 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f206a95a-d4fc-4e0d-a6bf-adf25f47a5e7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:00:50.648669 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f206a95a-d4fc-4e0d-a6bf-adf25f47a5e7 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:00:50.688780 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-f206a95a-d4fc-4e0d-a6bf-adf25f47a5e7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:00:50.688780 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:00:50.688780 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:00:50.688780 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:00:50.688780 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:00:50.688780 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:00:50.688780 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:00:50.688780 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:00:50.688780 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:00:50.688780 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:00:50.688780 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:00:50.688780 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:00:50.688780 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 12:00:50.688780 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 12:00:50.688780 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 12:00:50.690182 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 12:00:50.690182 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:00:50.690182 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:00:50.690182 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:00:50.690182 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:00:50.690182 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:00:50.690182 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:00:50.690182 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:00:50.690182 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jul 25 12:00:50.690182 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:00:50.690182 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:00:50.690182 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:00:50.690182 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:00:50.690182 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:00:50.690182 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:00:50.690182 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:00:50.690182 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:00:50.690971 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 12:00:50.690971 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 12:00:50.690971 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:00:50.690971 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jul 25 12:00:50.692953 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 12/27] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jul 25 12:00:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 405 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:00:50.694191 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f206a95a-d4fc-4e0d-a6bf-adf25f47a5e7 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:00:50.730324 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 16/28] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 25 12:00:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 12:00:50.731151 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d084560e-597f-48a9-a2d4-65b70d443d96 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:00:50.815951 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f206a95a-d4fc-4e0d-a6bf-adf25f47a5e7 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:00:50.842428 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d084560e-597f-48a9-a2d4-65b70d443d96 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:00:50.853536 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373d2724-803f-40de-a1bb-73f0e4eea3ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:00:50.854772 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373d2724-803f-40de-a1bb-73f0e4eea3ca None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:00:50.855236 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373d2724-803f-40de-a1bb-73f0e4eea3ca None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:00:50.855718 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373d2724-803f-40de-a1bb-73f0e4eea3ca None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:00:50.857373 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:00:50.870776 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373d2724-803f-40de-a1bb-73f0e4eea3ca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 12:00:50.880055 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9028df-fa67-4fb9-95d4-13c25fff5883 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:00:50.880728 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9028df-fa67-4fb9-95d4-13c25fff5883 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:00:50.881020 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9028df-fa67-4fb9-95d4-13c25fff5883 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:00:50.881307 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9028df-fa67-4fb9-95d4-13c25fff5883 None admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:00:50.882047 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9028df-fa67-4fb9-95d4-13c25fff5883 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:00:50.882361 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9028df-fa67-4fb9-95d4-13c25fff5883 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:00:50.882683 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-373d2724-803f-40de-a1bb-73f0e4eea3ca None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 12:00:50.882683 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:00:50.882683 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:00:50.882683 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:00:50.882683 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:00:50.882683 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:00:50.882683 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:00:50.882683 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:00:50.882683 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:00:50.882683 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:00:50.882683 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:00:50.882683 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:00:50.882683 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 12:00:50.882683 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 12:00:50.882683 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 12:00:50.882683 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 12:00:50.883524 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 12:00:50.883524 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 12:00:50.883524 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 12:00:50.883524 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 12:00:50.883524 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:00:50.883524 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:00:50.883524 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:00:50.883524 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:00:50.883524 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:00:50.883524 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:00:50.883524 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:00:50.883524 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jul 25 12:00:50.883524 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:00:50.883524 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:00:50.883524 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:00:50.883524 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:00:50.884298 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:00:50.884298 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:00:50.884298 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:00:50.884298 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:00:50.884298 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 12:00:50.884298 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 12:00:50.884298 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 12:00:50.884298 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 12:00:50.884298 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 12:00:50.884298 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 12:00:50.884298 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 12:00:50.884298 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jul 25 12:00:50.884932 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9028df-fa67-4fb9-95d4-13c25fff5883 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:00:50.886430 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 13/29] 10.222.0.42 () {56 vars in 1096 bytes} [Mon Jul 25 12:00:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:00:50.888003 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ba12b0-a596-43da-8dfa-8905ce8be9f1 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:00:50.888437 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ba12b0-a596-43da-8dfa-8905ce8be9f1 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:00:50.888923 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ba12b0-a596-43da-8dfa-8905ce8be9f1 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:00:50.890330 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 14/30] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 25 12:00:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:00:50.891260 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ba12b0-a596-43da-8dfa-8905ce8be9f1 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:00:50.897568 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 17/31] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jul 25 12:00:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:00:50.898468 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9028df-fa67-4fb9-95d4-13c25fff5883 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:00:50.949459 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:00:51.007372 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9028df-fa67-4fb9-95d4-13c25fff5883 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:00:51.008740 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ba12b0-a596-43da-8dfa-8905ce8be9f1 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:00:51.044712 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e129488-35c6-4417-8898-b524e4c4a535 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:00:51.045393 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e129488-35c6-4417-8898-b524e4c4a535 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:00:51.045673 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e129488-35c6-4417-8898-b524e4c4a535 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:00:51.045962 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e129488-35c6-4417-8898-b524e4c4a535 None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:00:51.046724 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e129488-35c6-4417-8898-b524e4c4a535 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:00:51.047042 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e129488-35c6-4417-8898-b524e4c4a535 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:00:51.048962 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dda8e56-74dd-4cd9-86ba-60c75d535e68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:00:51.049080 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e129488-35c6-4417-8898-b524e4c4a535 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:00:51.049969 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dda8e56-74dd-4cd9-86ba-60c75d535e68 None admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:00:51.050518 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dda8e56-74dd-4cd9-86ba-60c75d535e68 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:00:51.051100 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dda8e56-74dd-4cd9-86ba-60c75d535e68 None admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:00:51.052138 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dda8e56-74dd-4cd9-86ba-60c75d535e68 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:00:51.053853 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dda8e56-74dd-4cd9-86ba-60c75d535e68 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:00:51.057742 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 18/32] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Jul 25 12:00:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:00:51.059213 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8148fef-6b3b-409e-9254-2e1c3c596caf None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:00:51.059512 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8148fef-6b3b-409e-9254-2e1c3c596caf None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:00:51.059817 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8148fef-6b3b-409e-9254-2e1c3c596caf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:00:51.080353 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-c8148fef-6b3b-409e-9254-2e1c3c596caf None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:00:51.081558 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-c8148fef-6b3b-409e-9254-2e1c3c596caf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:00:51.112497 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:00:51.158878 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 19/33] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 25 12:00:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 12:00:51.159673 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8148fef-6b3b-409e-9254-2e1c3c596caf None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:00:51.274646 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8148fef-6b3b-409e-9254-2e1c3c596caf None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:00:51.306327 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 15/34] 10.222.0.42 () {60 vars in 1131 bytes} [Mon Jul 25 12:00:50 2022] POST /identity/v3/roles => generated 219 bytes in 415 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:00:51.307327 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dda8e56-74dd-4cd9-86ba-60c75d535e68 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:00:51.313426 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427fa826-8bcd-4905-bc28-07bf107730f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:00:51.315123 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427fa826-8bcd-4905-bc28-07bf107730f8 None admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:00:51.315723 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427fa826-8bcd-4905-bc28-07bf107730f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:00:51.316186 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427fa826-8bcd-4905-bc28-07bf107730f8 None admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:00:51.317724 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427fa826-8bcd-4905-bc28-07bf107730f8 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:00:51.319583 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427fa826-8bcd-4905-bc28-07bf107730f8 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:00:51.442788 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dda8e56-74dd-4cd9-86ba-60c75d535e68 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:00:51.454126 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:00:51.486730 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af28c92e-000e-446a-b00a-35f5154acf15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:00:51.487763 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 20/35] 10.222.0.42 () {60 vars in 1141 bytes} [Mon Jul 25 12:00:51 2022] POST /identity/v3/projects => generated 301 bytes in 329 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:00:51.488092 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af28c92e-000e-446a-b00a-35f5154acf15 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:00:51.488528 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af28c92e-000e-446a-b00a-35f5154acf15 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:00:51.488739 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427fa826-8bcd-4905-bc28-07bf107730f8 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:00:51.489185 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af28c92e-000e-446a-b00a-35f5154acf15 None admin] PATH_INFO: `/v3/users/0fad2691baf0472d99015f9f123f8d32` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:00:51.494219 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af28c92e-000e-446a-b00a-35f5154acf15 None admin] RBAC: Authorizing `identity:get_user(user_id=0fad2691baf0472d99015f9f123f8d32)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:00:51.495894 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af28c92e-000e-446a-b00a-35f5154acf15 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:00:51.497625 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 16/36] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Jul 25 12:00:51 2022] GET /identity/v3/users/0fad2691baf0472d99015f9f123f8d32 => generated 247 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:00:51.499174 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae845c68-ff82-4002-bd73-d363cccba848 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:00:51.499463 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae845c68-ff82-4002-bd73-d363cccba848 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:00:51.504053 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae845c68-ff82-4002-bd73-d363cccba848 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:00:51.506119 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 17/37] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 25 12:00:51 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:00:51.507432 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae845c68-ff82-4002-bd73-d363cccba848 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:00:51.511217 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:00:51.619299 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427fa826-8bcd-4905-bc28-07bf107730f8 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:00:51.647768 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae845c68-ff82-4002-bd73-d363cccba848 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:00:51.668570 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de9d85d-2bb4-406b-b36c-6ebd17b4f7af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:00:51.669278 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de9d85d-2bb4-406b-b36c-6ebd17b4f7af None admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:00:51.669602 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de9d85d-2bb4-406b-b36c-6ebd17b4f7af None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:00:51.669895 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de9d85d-2bb4-406b-b36c-6ebd17b4f7af None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:00:51.670675 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de9d85d-2bb4-406b-b36c-6ebd17b4f7af None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:00:51.672328 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de9d85d-2bb4-406b-b36c-6ebd17b4f7af None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:00:51.693843 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 21/38] 10.222.0.42 () {60 vars in 1131 bytes} [Mon Jul 25 12:00:51 2022] POST /identity/v3/roles => generated 225 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:00:51.694286 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be681d87-e52d-4f84-8cb7-44db769b9a6a None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:00:51.694588 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be681d87-e52d-4f84-8cb7-44db769b9a6a None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:00:51.694878 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be681d87-e52d-4f84-8cb7-44db769b9a6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:00:51.696242 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b516811-39c4-4079-9710-3ff9115b8536 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:00:51.697259 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b516811-39c4-4079-9710-3ff9115b8536 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:00:51.697557 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b516811-39c4-4079-9710-3ff9115b8536 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:00:51.697845 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b516811-39c4-4079-9710-3ff9115b8536 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:00:51.699262 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b516811-39c4-4079-9710-3ff9115b8536 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:00:51.701063 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b516811-39c4-4079-9710-3ff9115b8536 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:00:51.702929 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 18/39] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jul 25 12:00:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:00:51.705723 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b313d25a-49fb-49b3-a430-30cb6a14290d None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:00:51.706024 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b313d25a-49fb-49b3-a430-30cb6a14290d None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:00:51.706313 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b313d25a-49fb-49b3-a430-30cb6a14290d None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:00:51.707810 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 19/40] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 25 12:00:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:00:51.712917 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b313d25a-49fb-49b3-a430-30cb6a14290d None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:00:51.732248 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-be681d87-e52d-4f84-8cb7-44db769b9a6a None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:00:51.733872 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-be681d87-e52d-4f84-8cb7-44db769b9a6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:00:51.778503 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:00:51.826938 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 22/41] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 25 12:00:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 12:00:51.828779 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9b7f87-5659-4a92-98a5-9c23b33983f9 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:00:51.829180 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9b7f87-5659-4a92-98a5-9c23b33983f9 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:00:51.829621 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9b7f87-5659-4a92-98a5-9c23b33983f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:00:51.856919 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b313d25a-49fb-49b3-a430-30cb6a14290d None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:00:51.858341 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-cf9b7f87-5659-4a92-98a5-9c23b33983f9 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:00:51.859617 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-cf9b7f87-5659-4a92-98a5-9c23b33983f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:00:51.862553 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:00:51.901198 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e9364f-a5de-4eb4-9cd3-7e02a25674b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:00:51.902313 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e9364f-a5de-4eb4-9cd3-7e02a25674b5 None admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:00:51.902609 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e9364f-a5de-4eb4-9cd3-7e02a25674b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:00:51.902900 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e9364f-a5de-4eb4-9cd3-7e02a25674b5 None admin] PATH_INFO: `/v3/domains/default/users/0fad2691baf0472d99015f9f123f8d32/roles/a7d20885b6d94928a34b3cc5b3a00006` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:00:51.904989 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e9364f-a5de-4eb4-9cd3-7e02a25674b5 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=0fad2691baf0472d99015f9f123f8d32, role_id=a7d20885b6d94928a34b3cc5b3a00006)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:00:51.906485 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e9364f-a5de-4eb4-9cd3-7e02a25674b5 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:00:51.924695 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 20/42] 10.222.0.42 () {58 vars in 1361 bytes} [Mon Jul 25 12:00:51 2022] PUT /identity/v3/domains/default/users/0fad2691baf0472d99015f9f123f8d32/roles/a7d20885b6d94928a34b3cc5b3a00006 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:00:51.925513 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e9364f-a5de-4eb4-9cd3-7e02a25674b5 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:00:51.951150 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 23/43] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 25 12:00:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 12:00:51.976047 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9b7f87-5659-4a92-98a5-9c23b33983f9 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:00:52.044002 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e9364f-a5de-4eb4-9cd3-7e02a25674b5 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:00:52.047837 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:00:52.090780 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20fa1e8-d88b-4478-85bb-0da05a01a76a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:00:52.091470 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20fa1e8-d88b-4478-85bb-0da05a01a76a None admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:00:52.091798 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20fa1e8-d88b-4478-85bb-0da05a01a76a None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:00:52.092092 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20fa1e8-d88b-4478-85bb-0da05a01a76a None admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:00:52.092842 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20fa1e8-d88b-4478-85bb-0da05a01a76a None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:00:52.094266 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20fa1e8-d88b-4478-85bb-0da05a01a76a None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:00:52.094916 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9b7f87-5659-4a92-98a5-9c23b33983f9 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:00:52.111757 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 21/44] 10.222.0.42 () {60 vars in 1131 bytes} [Mon Jul 25 12:00:51 2022] POST /identity/v3/roles => generated 223 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:00:52.139427 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff05cde8-5b68-4cf9-b45b-b4c069150c42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:00:52.140707 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff05cde8-5b68-4cf9-b45b-b4c069150c42 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:00:52.141253 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff05cde8-5b68-4cf9-b45b-b4c069150c42 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:00:52.141734 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff05cde8-5b68-4cf9-b45b-b4c069150c42 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:00:52.143286 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff05cde8-5b68-4cf9-b45b-b4c069150c42 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:00:52.145212 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff05cde8-5b68-4cf9-b45b-b4c069150c42 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:00:52.147242 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 24/45] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jul 25 12:00:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 12:00:52.153470 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20fa1e8-d88b-4478-85bb-0da05a01a76a None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:00:52.271607 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20fa1e8-d88b-4478-85bb-0da05a01a76a None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:00:52.312418 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c35eba6-0a21-43db-ad13-d6b0826ceac4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:00:52.313123 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c35eba6-0a21-43db-ad13-d6b0826ceac4 None admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:00:52.313388 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c35eba6-0a21-43db-ad13-d6b0826ceac4 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:00:52.313678 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c35eba6-0a21-43db-ad13-d6b0826ceac4 None admin] PATH_INFO: `/v3/projects` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:00:52.314437 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c35eba6-0a21-43db-ad13-d6b0826ceac4 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:00:52.315946 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c35eba6-0a21-43db-ad13-d6b0826ceac4 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:00:52.344778 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 22/46] 10.222.0.42 () {60 vars in 1141 bytes} [Mon Jul 25 12:00:52 2022] POST /identity/v3/projects => generated 312 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:00:53.944403 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075ddf8b-5f61-4335-8625-7f3cb6dcc171 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:00:53.944906 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075ddf8b-5f61-4335-8625-7f3cb6dcc171 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:00:53.945011 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075ddf8b-5f61-4335-8625-7f3cb6dcc171 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:00:53.946344 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 25/47] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 25 12:00:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:00:53.951445 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba123e43-99f8-4240-adb5-4fc6677f66a9 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:00:53.951875 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba123e43-99f8-4240-adb5-4fc6677f66a9 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:00:53.952055 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba123e43-99f8-4240-adb5-4fc6677f66a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:00:53.971448 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-ba123e43-99f8-4240-adb5-4fc6677f66a9 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:00:53.972997 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-ba123e43-99f8-4240-adb5-4fc6677f66a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:00:53.983592 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:00:54.040374 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 23/48] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 25 12:00:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 12:00:54.063013 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075ddf8b-5f61-4335-8625-7f3cb6dcc171 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:00:54.071948 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:00:54.164479 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075ddf8b-5f61-4335-8625-7f3cb6dcc171 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:00:54.198212 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45c678a-ca58-4d57-8102-80b723a4bbc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:00:54.199009 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45c678a-ca58-4d57-8102-80b723a4bbc2 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:00:54.199301 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45c678a-ca58-4d57-8102-80b723a4bbc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:00:54.199614 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45c678a-ca58-4d57-8102-80b723a4bbc2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:00:54.210196 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45c678a-ca58-4d57-8102-80b723a4bbc2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 12:00:54.217749 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-a45c678a-ca58-4d57-8102-80b723a4bbc2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 12:00:54.217749 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:00:54.217749 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:00:54.217749 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:00:54.217749 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:00:54.217749 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:00:54.217749 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:00:54.217749 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:00:54.217749 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:00:54.217749 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:00:54.217749 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:00:54.217749 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:00:54.217749 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 12:00:54.217749 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 12:00:54.217749 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 12:00:54.217749 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 12:00:54.218593 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 12:00:54.218593 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 12:00:54.218593 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 12:00:54.218593 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 12:00:54.218593 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:00:54.218593 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:00:54.218593 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:00:54.218593 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:00:54.218593 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:00:54.218593 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:00:54.218593 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:00:54.218593 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 25 12:00:54.218593 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:00:54.218593 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:00:54.218593 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:00:54.218593 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:00:54.219403 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:00:54.219403 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:00:54.219403 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:00:54.219403 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:00:54.219403 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 12:00:54.219403 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 12:00:54.219403 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 12:00:54.219403 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 12:00:54.219403 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 12:00:54.219403 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 12:00:54.219403 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 12:00:54.219403 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 25 12:00:54.219403 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 26/49] 10.222.0.42 () {56 vars in 1096 bytes} [Mon Jul 25 12:00:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:00:54.225450 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba123e43-99f8-4240-adb5-4fc6677f66a9 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:00:54.340602 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba123e43-99f8-4240-adb5-4fc6677f66a9 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:00:54.355697 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653641ef-0089-40f6-9a05-46b4fc87eef0 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:00:54.356086 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653641ef-0089-40f6-9a05-46b4fc87eef0 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:00:54.356444 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653641ef-0089-40f6-9a05-46b4fc87eef0 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:00:54.357911 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 27/50] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 25 12:00:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 12:00:54.362808 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e74e374-098b-4b8e-9357-6ba6faeef10c None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:00:54.363103 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e74e374-098b-4b8e-9357-6ba6faeef10c None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:00:54.363392 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e74e374-098b-4b8e-9357-6ba6faeef10c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:00:54.375562 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9885cd7e-efbe-4ddf-97c2-54b5dc58fc03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:00:54.376246 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9885cd7e-efbe-4ddf-97c2-54b5dc58fc03 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:00:54.376542 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9885cd7e-efbe-4ddf-97c2-54b5dc58fc03 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:00:54.376834 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9885cd7e-efbe-4ddf-97c2-54b5dc58fc03 None admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:00:54.377597 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9885cd7e-efbe-4ddf-97c2-54b5dc58fc03 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:00:54.377914 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9885cd7e-efbe-4ddf-97c2-54b5dc58fc03 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:00:54.379301 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9885cd7e-efbe-4ddf-97c2-54b5dc58fc03 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:00:54.381149 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-8e74e374-098b-4b8e-9357-6ba6faeef10c None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:00:54.382347 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-8e74e374-098b-4b8e-9357-6ba6faeef10c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:00:54.390665 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 24/51] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Jul 25 12:00:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:00:54.397191 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9885cd7e-efbe-4ddf-97c2-54b5dc58fc03 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:00:54.407744 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:00:54.442580 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:00:54.451136 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 28/52] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 25 12:00:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 25 12:00:54.474541 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e74e374-098b-4b8e-9357-6ba6faeef10c None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:00:54.497720 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9885cd7e-efbe-4ddf-97c2-54b5dc58fc03 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:00:54.534099 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bb310d-62f0-49e6-84c5-57f91102fcb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:00:54.534980 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bb310d-62f0-49e6-84c5-57f91102fcb2 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:00:54.535276 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bb310d-62f0-49e6-84c5-57f91102fcb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:00:54.535583 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bb310d-62f0-49e6-84c5-57f91102fcb2 None admin] PATH_INFO: `/v3/users/0fad2691baf0472d99015f9f123f8d32` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:00:54.536803 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2bb310d-62f0-49e6-84c5-57f91102fcb2 None admin] RBAC: Authorizing `identity:get_user(user_id=0fad2691baf0472d99015f9f123f8d32)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:00:54.538266 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2bb310d-62f0-49e6-84c5-57f91102fcb2 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:00:54.540210 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 25/53] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Jul 25 12:00:54 2022] GET /identity/v3/users/0fad2691baf0472d99015f9f123f8d32 => generated 247 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 12:00:54.545604 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bb310d-62f0-49e6-84c5-57f91102fcb2 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:00:54.583754 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e74e374-098b-4b8e-9357-6ba6faeef10c None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:00:54.620315 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2c72a2-5b08-4932-8a86-c0d0341b77b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:00:54.621069 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2c72a2-5b08-4932-8a86-c0d0341b77b9 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:00:54.621359 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2c72a2-5b08-4932-8a86-c0d0341b77b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:00:54.621640 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2c72a2-5b08-4932-8a86-c0d0341b77b9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:00:54.622812 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2c72a2-5b08-4932-8a86-c0d0341b77b9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:00:54.624312 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2c72a2-5b08-4932-8a86-c0d0341b77b9 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:00:54.626126 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 29/54] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jul 25 12:00:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 12:00:54.631443 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2c72a2-5b08-4932-8a86-c0d0341b77b9 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:00:54.631947 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:00:54.655122 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bb310d-62f0-49e6-84c5-57f91102fcb2 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:00:54.690039 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c59283-ac3b-4f9a-ba8c-101988f9bde7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:00:54.690832 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c59283-ac3b-4f9a-ba8c-101988f9bde7 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:00:54.691127 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c59283-ac3b-4f9a-ba8c-101988f9bde7 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:00:54.691417 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c59283-ac3b-4f9a-ba8c-101988f9bde7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:00:54.692655 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c59283-ac3b-4f9a-ba8c-101988f9bde7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:00:54.693931 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:00:54.694117 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c59283-ac3b-4f9a-ba8c-101988f9bde7 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:00:54.696124 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 26/55] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jul 25 12:00:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 12:00:54.701588 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c59283-ac3b-4f9a-ba8c-101988f9bde7 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:00:54.735134 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2c72a2-5b08-4932-8a86-c0d0341b77b9 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:00:54.769093 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84f066c-6a2b-465b-b5cf-705b6252fb74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:00:54.769853 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84f066c-6a2b-465b-b5cf-705b6252fb74 None admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:00:54.770154 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84f066c-6a2b-465b-b5cf-705b6252fb74 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:00:54.770442 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84f066c-6a2b-465b-b5cf-705b6252fb74 None admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:00:54.771166 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84f066c-6a2b-465b-b5cf-705b6252fb74 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:00:54.772683 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84f066c-6a2b-465b-b5cf-705b6252fb74 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:00:54.801379 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 30/56] 10.222.0.42 () {60 vars in 1140 bytes} [Mon Jul 25 12:00:54 2022] POST /identity/v3/projects => generated 298 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 12:00:54.811453 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c59283-ac3b-4f9a-ba8c-101988f9bde7 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:00:54.849113 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e6bfae-9737-4289-8f63-96c14181cbb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:00:54.850092 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e6bfae-9737-4289-8f63-96c14181cbb8 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:00:54.850382 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e6bfae-9737-4289-8f63-96c14181cbb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:00:54.850668 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e6bfae-9737-4289-8f63-96c14181cbb8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:00:54.851538 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e6bfae-9737-4289-8f63-96c14181cbb8 None admin] RBAC: Adding query filter params (role.id=a7d20885b6d94928a34b3cc5b3a00006, scope.domain.id=default, user.id=0fad2691baf0472d99015f9f123f8d32) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:00:54.851894 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e6bfae-9737-4289-8f63-96c14181cbb8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:00:54.853416 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e6bfae-9737-4289-8f63-96c14181cbb8 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:00:54.860173 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 27/57] 10.222.0.42 () {56 vars in 1322 bytes} [Mon Jul 25 12:00:54 2022] GET /identity/v3/role_assignments?user.id=0fad2691baf0472d99015f9f123f8d32&scope.domain.id=default&role.id=a7d20885b6d94928a34b3cc5b3a00006 => generated 530 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 12:00:56.815595 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8ff341-9732-4ba0-b6f7-90462b394919 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:00:56.815996 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8ff341-9732-4ba0-b6f7-90462b394919 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:00:56.816183 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8ff341-9732-4ba0-b6f7-90462b394919 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:00:56.817574 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 31/58] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 25 12:00:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:00:56.822608 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e658206e-f9ba-402e-93ef-4824a567359f None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:00:56.822998 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e658206e-f9ba-402e-93ef-4824a567359f None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:00:56.823183 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e658206e-f9ba-402e-93ef-4824a567359f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:00:56.841546 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-e658206e-f9ba-402e-93ef-4824a567359f None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:00:56.842744 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-e658206e-f9ba-402e-93ef-4824a567359f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:00:56.844705 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:00:56.909154 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 28/59] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 25 12:00:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 12:00:56.932548 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8ff341-9732-4ba0-b6f7-90462b394919 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:00:57.041973 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8ff341-9732-4ba0-b6f7-90462b394919 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:00:57.057672 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:00:57.076721 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b124f926-7aa6-40f3-8b53-2106958c3b8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:00:57.077520 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b124f926-7aa6-40f3-8b53-2106958c3b8e None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:00:57.077809 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b124f926-7aa6-40f3-8b53-2106958c3b8e None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:00:57.078088 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b124f926-7aa6-40f3-8b53-2106958c3b8e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:00:57.079265 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b124f926-7aa6-40f3-8b53-2106958c3b8e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:00:57.080767 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b124f926-7aa6-40f3-8b53-2106958c3b8e None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:00:57.082368 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 32/60] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jul 25 12:00:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:00:57.087925 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e658206e-f9ba-402e-93ef-4824a567359f None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:00:57.184536 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e658206e-f9ba-402e-93ef-4824a567359f None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:00:57.217130 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e939cc5-a798-4e74-be5d-3d26563e5629 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:00:57.217788 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e939cc5-a798-4e74-be5d-3d26563e5629 None admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:00:57.218077 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e939cc5-a798-4e74-be5d-3d26563e5629 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:00:57.218359 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e939cc5-a798-4e74-be5d-3d26563e5629 None admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:00:57.219080 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e939cc5-a798-4e74-be5d-3d26563e5629 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:00:57.220581 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e939cc5-a798-4e74-be5d-3d26563e5629 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:00:57.234088 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:00:57.248344 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 29/61] 10.222.0.42 () {60 vars in 1132 bytes} [Mon Jul 25 12:00:57 2022] POST /identity/v3/users => generated 275 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:01:00.330590 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4202a1-db7e-4ca1-a5b7-92c3508d2adf None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:00.331674 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4202a1-db7e-4ca1-a5b7-92c3508d2adf None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:00.332172 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4202a1-db7e-4ca1-a5b7-92c3508d2adf None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:00.333761 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 33/62] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 25 12:01:00 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:01:00.339099 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d9096c-7826-4caf-9ca1-05695ab6f5e4 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:00.339946 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d9096c-7826-4caf-9ca1-05695ab6f5e4 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:00.340449 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d9096c-7826-4caf-9ca1-05695ab6f5e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:00.363408 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be9679c-3d3e-4810-80bd-0c926becbabd None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:00.364064 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be9679c-3d3e-4810-80bd-0c926becbabd None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:00.364552 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be9679c-3d3e-4810-80bd-0c926becbabd None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:00.366121 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 34/63] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 25 12:01:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 12:01:00.367176 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-76d9096c-7826-4caf-9ca1-05695ab6f5e4 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:01:00.368637 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-76d9096c-7826-4caf-9ca1-05695ab6f5e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:01:00.375818 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222fc599-0443-476f-893a-b0b140d6ce3d None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:00.376372 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222fc599-0443-476f-893a-b0b140d6ce3d None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:00.376897 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222fc599-0443-476f-893a-b0b140d6ce3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:00.401327 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-222fc599-0443-476f-893a-b0b140d6ce3d None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:01:00.403323 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-222fc599-0443-476f-893a-b0b140d6ce3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:01:00.513869 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 35/64] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 25 12:01:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 25 12:01:00.526167 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 30/65] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 25 12:01:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 12:01:00.537283 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222fc599-0443-476f-893a-b0b140d6ce3d None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:00.561050 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d9096c-7826-4caf-9ca1-05695ab6f5e4 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:00.653919 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222fc599-0443-476f-893a-b0b140d6ce3d None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:00.671159 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:00.694531 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bb0402-7bfa-41dd-aa5d-1fa7f4bc25c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:00.695815 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bb0402-7bfa-41dd-aa5d-1fa7f4bc25c6 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:00.696316 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bb0402-7bfa-41dd-aa5d-1fa7f4bc25c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:00.696822 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bb0402-7bfa-41dd-aa5d-1fa7f4bc25c6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:00.706410 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d9096c-7826-4caf-9ca1-05695ab6f5e4 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:00.711243 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bb0402-7bfa-41dd-aa5d-1fa7f4bc25c6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 12:01:00.721238 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-d8bb0402-7bfa-41dd-aa5d-1fa7f4bc25c6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 12:01:00.721238 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:01:00.721238 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:01:00.721238 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:01:00.721238 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:01:00.721238 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:01:00.721238 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:01:00.721238 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:01:00.721238 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:01:00.721238 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:01:00.721238 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:01:00.721238 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:01:00.721238 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 12:01:00.721238 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 12:01:00.721238 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 12:01:00.721238 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 12:01:00.722100 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 12:01:00.722100 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 12:01:00.722100 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 12:01:00.722100 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 12:01:00.722100 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:01:00.722100 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:01:00.722100 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:01:00.722100 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:01:00.722100 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:01:00.722100 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:01:00.722100 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:01:00.722100 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 25 12:01:00.722100 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:01:00.722100 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:01:00.722100 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:01:00.722100 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:01:00.722932 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:01:00.722932 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:01:00.722932 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:01:00.722932 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:01:00.722932 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 12:01:00.722932 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 12:01:00.722932 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 12:01:00.722932 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 12:01:00.722932 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 12:01:00.722932 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 12:01:00.722932 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 12:01:00.722932 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 25 12:01:00.722932 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 36/66] 10.222.0.42 () {56 vars in 1096 bytes} [Mon Jul 25 12:01:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:01:00.727591 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d58878c-46d7-47d4-96bb-f7e31340356f None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:00.728097 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d58878c-46d7-47d4-96bb-f7e31340356f None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:00.728557 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d58878c-46d7-47d4-96bb-f7e31340356f None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:00.730070 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 37/67] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 25 12:01:00 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:01:00.731754 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d58878c-46d7-47d4-96bb-f7e31340356f None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:00.751941 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:00.758392 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa9257f-680d-4d5a-a438-dd3f527610db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:00.759659 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa9257f-680d-4d5a-a438-dd3f527610db None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:00.759928 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa9257f-680d-4d5a-a438-dd3f527610db None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:00.761078 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa9257f-680d-4d5a-a438-dd3f527610db None admin] PATH_INFO: `/v3/roles/member` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:00.784726 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa9257f-680d-4d5a-a438-dd3f527610db None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 12:01:00.793195 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-caa9257f-680d-4d5a-a438-dd3f527610db None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 25 12:01:00.793195 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:01:00.793195 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:01:00.793195 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:01:00.793195 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:01:00.793195 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:01:00.793195 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:01:00.793195 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:01:00.793195 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:01:00.793195 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:01:00.793195 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:01:00.793195 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:01:00.793195 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 12:01:00.793195 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 12:01:00.793195 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 12:01:00.793195 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 12:01:00.794043 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 12:01:00.794043 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 12:01:00.794043 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 12:01:00.794043 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 12:01:00.794043 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:01:00.794043 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:01:00.794043 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:01:00.794043 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:01:00.794043 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:01:00.794043 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:01:00.794043 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:01:00.794043 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jul 25 12:01:00.794043 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:01:00.794043 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:01:00.794043 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:01:00.794043 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:01:00.794824 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:01:00.794824 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:01:00.794824 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:01:00.794824 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:01:00.794824 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 12:01:00.794824 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 12:01:00.794824 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 12:01:00.794824 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 12:01:00.794824 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 12:01:00.794824 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 12:01:00.794824 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 25 12:01:00.794824 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jul 25 12:01:00.794824 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 31/68] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Jul 25 12:01:00 2022] GET /identity/v3/roles/member => generated 84 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:01:00.796157 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb689c05-dcd3-449c-9828-82009dbbb8f0 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:00.796456 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb689c05-dcd3-449c-9828-82009dbbb8f0 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:00.796748 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb689c05-dcd3-449c-9828-82009dbbb8f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:00.816813 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-bb689c05-dcd3-449c-9828-82009dbbb8f0 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:01:00.818412 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-bb689c05-dcd3-449c-9828-82009dbbb8f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:01:00.862552 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d58878c-46d7-47d4-96bb-f7e31340356f None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:00.898969 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 32/69] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 25 12:01:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 12:01:00.899168 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f689cb-49b3-4ffe-a704-47042861b6c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:00.900195 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f689cb-49b3-4ffe-a704-47042861b6c5 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:00.900307 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb689c05-dcd3-449c-9828-82009dbbb8f0 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:00.900762 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f689cb-49b3-4ffe-a704-47042861b6c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:00.901242 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f689cb-49b3-4ffe-a704-47042861b6c5 None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:00.902180 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f689cb-49b3-4ffe-a704-47042861b6c5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:01:00.902764 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f689cb-49b3-4ffe-a704-47042861b6c5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:00.904531 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f689cb-49b3-4ffe-a704-47042861b6c5 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:00.914686 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 38/70] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Jul 25 12:01:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:01:00.921759 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2aae3e-e8da-485e-9a70-d5e9bab69227 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:00.922056 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2aae3e-e8da-485e-9a70-d5e9bab69227 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:00.922342 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2aae3e-e8da-485e-9a70-d5e9bab69227 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:00.923959 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 39/71] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 25 12:01:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 12:01:00.928930 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2aae3e-e8da-485e-9a70-d5e9bab69227 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:00.966042 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:01.013837 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb689c05-dcd3-449c-9828-82009dbbb8f0 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:01.028394 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:01.030933 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2aae3e-e8da-485e-9a70-d5e9bab69227 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:01.050137 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbe6411-44f4-469b-b681-b62597ae044b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:01.050813 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbe6411-44f4-469b-b681-b62597ae044b None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:01.051107 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbe6411-44f4-469b-b681-b62597ae044b None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:01.051391 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbe6411-44f4-469b-b681-b62597ae044b None admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:01.052637 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbe6411-44f4-469b-b681-b62597ae044b None admin] RBAC: Adding query filter params (name=member) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:01:01.052963 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbe6411-44f4-469b-b681-b62597ae044b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:01.054362 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbe6411-44f4-469b-b681-b62597ae044b None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:01.062416 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 33/72] 10.222.0.42 () {56 vars in 1101 bytes} [Mon Jul 25 12:01:00 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:01:01.063598 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f138a4ed-f37b-48fe-b76e-bfd1d1a6c660 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:01.063925 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f138a4ed-f37b-48fe-b76e-bfd1d1a6c660 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:01.064216 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f138a4ed-f37b-48fe-b76e-bfd1d1a6c660 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:01.065871 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83099f97-edba-46a2-84a7-393bb5731dcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:01.066786 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83099f97-edba-46a2-84a7-393bb5731dcf None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:01.067049 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83099f97-edba-46a2-84a7-393bb5731dcf None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:01.067336 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83099f97-edba-46a2-84a7-393bb5731dcf None admin] PATH_INFO: `/v3/users/0fad2691baf0472d99015f9f123f8d32` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:01.068583 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83099f97-edba-46a2-84a7-393bb5731dcf None admin] RBAC: Authorizing `identity:get_user(user_id=0fad2691baf0472d99015f9f123f8d32)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:01.070033 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83099f97-edba-46a2-84a7-393bb5731dcf None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:01.071690 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 40/73] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Jul 25 12:01:00 2022] GET /identity/v3/users/0fad2691baf0472d99015f9f123f8d32 => generated 247 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:01:01.072459 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83099f97-edba-46a2-84a7-393bb5731dcf None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:01.082504 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-f138a4ed-f37b-48fe-b76e-bfd1d1a6c660 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:01:01.083754 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-f138a4ed-f37b-48fe-b76e-bfd1d1a6c660 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:01:01.152075 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 34/74] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 25 12:01:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 12:01:01.153396 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77573614-1e02-43df-8948-ef30a8c9381d None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:01.153695 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77573614-1e02-43df-8948-ef30a8c9381d None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:01.153984 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77573614-1e02-43df-8948-ef30a8c9381d None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:01.155445 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 35/75] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 25 12:01:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:01:01.155996 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77573614-1e02-43df-8948-ef30a8c9381d None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:01.185495 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83099f97-edba-46a2-84a7-393bb5731dcf None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:01.188531 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:01.221545 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77825e73-12ec-486d-8c1c-f52e55f001c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:01.222349 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77825e73-12ec-486d-8c1c-f52e55f001c9 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:01.222640 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77825e73-12ec-486d-8c1c-f52e55f001c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:01.222923 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77825e73-12ec-486d-8c1c-f52e55f001c9 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:01.232711 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77825e73-12ec-486d-8c1c-f52e55f001c9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 12:01:01.240760 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-77825e73-12ec-486d-8c1c-f52e55f001c9 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 12:01:01.240760 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:01:01.240760 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:01:01.240760 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:01:01.240760 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:01:01.240760 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:01:01.240760 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:01:01.240760 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:01:01.240760 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:01:01.240760 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:01:01.240760 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:01:01.240760 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:01:01.240760 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 12:01:01.240760 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 12:01:01.240760 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 12:01:01.240760 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 12:01:01.241587 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 12:01:01.241587 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 12:01:01.241587 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 12:01:01.241587 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 12:01:01.241587 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:01:01.241587 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:01:01.241587 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:01:01.241587 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:01:01.241587 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:01:01.241587 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:01:01.241587 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:01:01.241587 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 25 12:01:01.241587 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:01:01.241587 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:01:01.241587 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:01:01.241587 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:01:01.247286 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:01:01.247286 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:01:01.247286 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:01:01.247286 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:01:01.247286 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 12:01:01.247286 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 12:01:01.247286 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 12:01:01.247286 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 12:01:01.247286 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 12:01:01.247286 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 12:01:01.247286 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 12:01:01.247286 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 25 12:01:01.247286 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 41/76] 10.222.0.42 () {56 vars in 1114 bytes} [Mon Jul 25 12:01:01 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:01:01.247286 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77825e73-12ec-486d-8c1c-f52e55f001c9 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:01.257985 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77573614-1e02-43df-8948-ef30a8c9381d None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:01.265248 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:01.292524 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0c2ff0-703d-469d-af99-625386ed36a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:01.293394 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0c2ff0-703d-469d-af99-625386ed36a9 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:01.293679 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0c2ff0-703d-469d-af99-625386ed36a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:01.293957 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0c2ff0-703d-469d-af99-625386ed36a9 None admin] PATH_INFO: `/v3/users/b66651fed8eb43758d5d02472aa1ffaf` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:01.308472 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0c2ff0-703d-469d-af99-625386ed36a9 None admin] RBAC: Authorizing `identity:get_user(user_id=b66651fed8eb43758d5d02472aa1ffaf)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:01.309907 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0c2ff0-703d-469d-af99-625386ed36a9 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:01.311570 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 36/77] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Jul 25 12:01:01 2022] GET /identity/v3/users/b66651fed8eb43758d5d02472aa1ffaf => generated 275 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:01:01.312742 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf89b06-e8e4-4f2e-bbeb-a0c93f79026c None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:01.313029 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf89b06-e8e4-4f2e-bbeb-a0c93f79026c None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:01.313311 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf89b06-e8e4-4f2e-bbeb-a0c93f79026c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:01.330961 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-adf89b06-e8e4-4f2e-bbeb-a0c93f79026c None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:01:01.332177 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-adf89b06-e8e4-4f2e-bbeb-a0c93f79026c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:01:01.348376 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77825e73-12ec-486d-8c1c-f52e55f001c9 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:01.380901 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a365747-8a96-499c-b55d-8e5e81b15f8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:01.381633 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a365747-8a96-499c-b55d-8e5e81b15f8b None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:01.381937 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a365747-8a96-499c-b55d-8e5e81b15f8b None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:01.382225 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a365747-8a96-499c-b55d-8e5e81b15f8b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:01.400915 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 37/78] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 25 12:01:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 12:01:01.401784 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf89b06-e8e4-4f2e-bbeb-a0c93f79026c None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:01.416589 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:01.450278 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a365747-8a96-499c-b55d-8e5e81b15f8b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:01.451805 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a365747-8a96-499c-b55d-8e5e81b15f8b None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:01.503788 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:01.510406 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf89b06-e8e4-4f2e-bbeb-a0c93f79026c None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:01.544319 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c47a4e7-47b3-498b-b22a-6b32b59fc7af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:01.545141 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c47a4e7-47b3-498b-b22a-6b32b59fc7af None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:01.545432 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c47a4e7-47b3-498b-b22a-6b32b59fc7af None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:01.545713 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c47a4e7-47b3-498b-b22a-6b32b59fc7af None admin] PATH_INFO: `/v3/domains/default` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:01.546883 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c47a4e7-47b3-498b-b22a-6b32b59fc7af None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:01.548867 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c47a4e7-47b3-498b-b22a-6b32b59fc7af None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:01.549331 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 42/79] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Jul 25 12:01:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 12:01:01.550174 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a365747-8a96-499c-b55d-8e5e81b15f8b None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:01.550467 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 38/80] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jul 25 12:01:01 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:01:01.550467 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c47a4e7-47b3-498b-b22a-6b32b59fc7af None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:01.616869 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:01.652966 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c47a4e7-47b3-498b-b22a-6b32b59fc7af None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:01.654849 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a365747-8a96-499c-b55d-8e5e81b15f8b None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:01.686897 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937232a6-d220-415d-8e45-104d85a44cff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:01.687489 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937232a6-d220-415d-8e45-104d85a44cff None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:01.687804 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937232a6-d220-415d-8e45-104d85a44cff None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:01.688092 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937232a6-d220-415d-8e45-104d85a44cff None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:01.688538 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f690394-0b37-4a1d-b6f2-6c0eaebd2e47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:01.689769 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f690394-0b37-4a1d-b6f2-6c0eaebd2e47 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:01.690063 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f690394-0b37-4a1d-b6f2-6c0eaebd2e47 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:01.690349 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f690394-0b37-4a1d-b6f2-6c0eaebd2e47 None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:01.691108 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f690394-0b37-4a1d-b6f2-6c0eaebd2e47 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:01:01.691426 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f690394-0b37-4a1d-b6f2-6c0eaebd2e47 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:01.692858 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f690394-0b37-4a1d-b6f2-6c0eaebd2e47 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:01.700828 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 43/81] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Jul 25 12:01:01 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:01:01.701614 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f690394-0b37-4a1d-b6f2-6c0eaebd2e47 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:01.718682 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:01.757542 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937232a6-d220-415d-8e45-104d85a44cff None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:01.759036 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937232a6-d220-415d-8e45-104d85a44cff None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:01.804929 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f690394-0b37-4a1d-b6f2-6c0eaebd2e47 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:01.830221 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:01.839085 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4a16aa-5acd-4c8b-84a3-09a9a74c2b17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:01.839916 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4a16aa-5acd-4c8b-84a3-09a9a74c2b17 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:01.840217 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4a16aa-5acd-4c8b-84a3-09a9a74c2b17 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:01.840511 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4a16aa-5acd-4c8b-84a3-09a9a74c2b17 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:01.851120 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4a16aa-5acd-4c8b-84a3-09a9a74c2b17 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 12:01:01.858649 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-ac4a16aa-5acd-4c8b-84a3-09a9a74c2b17 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 25 12:01:01.858649 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:01:01.858649 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:01:01.858649 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:01:01.858649 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:01:01.858649 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:01:01.858649 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:01:01.858649 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:01:01.858649 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:01:01.858649 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:01:01.858649 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:01:01.858649 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:01:01.858649 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 12:01:01.858649 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 12:01:01.858649 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 12:01:01.858649 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 12:01:01.859598 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 12:01:01.859598 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 12:01:01.859598 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 12:01:01.859598 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 12:01:01.859598 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:01:01.859598 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:01:01.859598 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:01:01.859598 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:01:01.859598 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:01:01.859598 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:01:01.859598 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:01:01.859598 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 25 12:01:01.859598 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:01:01.859598 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:01:01.859598 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:01:01.859598 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:01:01.860370 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:01:01.860370 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:01:01.860370 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:01:01.860370 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:01:01.860370 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 12:01:01.860370 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 12:01:01.860370 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 12:01:01.860370 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 12:01:01.860370 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 12:01:01.860370 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 12:01:01.860370 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 25 12:01:01.860370 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 25 12:01:01.860984 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 39/82] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Jul 25 12:01:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 12:01:01.861052 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 44/83] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Jul 25 12:01:01 2022] GET /identity/v3/roles/member => generated 84 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:01:01.861122 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937232a6-d220-415d-8e45-104d85a44cff None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:01.861187 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4a16aa-5acd-4c8b-84a3-09a9a74c2b17 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:01.920792 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:01.959853 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937232a6-d220-415d-8e45-104d85a44cff None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:01.962770 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4a16aa-5acd-4c8b-84a3-09a9a74c2b17 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:01.992859 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3cc3ac-a891-4d01-bdaa-30da24857309 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:01.993712 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3cc3ac-a891-4d01-bdaa-30da24857309 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:01.994003 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3cc3ac-a891-4d01-bdaa-30da24857309 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:01.994293 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3cc3ac-a891-4d01-bdaa-30da24857309 None admin] PATH_INFO: `/v3/projects/23c9d26ac78646c79204b87bd5332e2c` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:01.994754 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2834e8-af9d-4eb6-9416-7885b06969a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:01.995441 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2834e8-af9d-4eb6-9416-7885b06969a6 None admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:01.995539 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3cc3ac-a891-4d01-bdaa-30da24857309 None admin] RBAC: Authorizing `identity:get_project(project_id=23c9d26ac78646c79204b87bd5332e2c)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:01.995771 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2834e8-af9d-4eb6-9416-7885b06969a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:01.996063 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2834e8-af9d-4eb6-9416-7885b06969a6 None admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:01.996791 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2834e8-af9d-4eb6-9416-7885b06969a6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:01.996956 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3cc3ac-a891-4d01-bdaa-30da24857309 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:01.998242 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2834e8-af9d-4eb6-9416-7885b06969a6 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:01.998655 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 40/84] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Jul 25 12:01:01 2022] GET /identity/v3/projects/23c9d26ac78646c79204b87bd5332e2c => generated 298 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:01:01.999422 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3cc3ac-a891-4d01-bdaa-30da24857309 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:02.030533 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 45/85] 10.222.0.42 () {60 vars in 1132 bytes} [Mon Jul 25 12:01:01 2022] POST /identity/v3/users => generated 289 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:01:02.031244 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2834e8-af9d-4eb6-9416-7885b06969a6 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:02.084083 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:02.117264 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3cc3ac-a891-4d01-bdaa-30da24857309 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:02.150783 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2834e8-af9d-4eb6-9416-7885b06969a6 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:02.153032 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:02.163203 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdcdb91-c94f-49e4-88ce-6722dc9ff222 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:02.164757 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdcdb91-c94f-49e4-88ce-6722dc9ff222 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:02.165051 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdcdb91-c94f-49e4-88ce-6722dc9ff222 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:02.165336 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdcdb91-c94f-49e4-88ce-6722dc9ff222 None admin] PATH_INFO: `/v3/users/b66651fed8eb43758d5d02472aa1ffaf` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:02.166656 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdcdb91-c94f-49e4-88ce-6722dc9ff222 None admin] RBAC: Authorizing `identity:get_user(user_id=b66651fed8eb43758d5d02472aa1ffaf)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:02.168285 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdcdb91-c94f-49e4-88ce-6722dc9ff222 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:02.175155 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 41/86] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Jul 25 12:01:01 2022] GET /identity/v3/users/b66651fed8eb43758d5d02472aa1ffaf => generated 275 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:01:02.175155 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdcdb91-c94f-49e4-88ce-6722dc9ff222 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:02.197895 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56f2365-52d3-4c3e-9e54-a5b3ae000fe5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:02.198777 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56f2365-52d3-4c3e-9e54-a5b3ae000fe5 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:02.199071 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56f2365-52d3-4c3e-9e54-a5b3ae000fe5 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:02.199354 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56f2365-52d3-4c3e-9e54-a5b3ae000fe5 None admin] PATH_INFO: `/v3/projects/23c9d26ac78646c79204b87bd5332e2c` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:02.201925 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56f2365-52d3-4c3e-9e54-a5b3ae000fe5 None admin] RBAC: Authorizing `identity:get_project(project_id=23c9d26ac78646c79204b87bd5332e2c)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:02.203804 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56f2365-52d3-4c3e-9e54-a5b3ae000fe5 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:02.205251 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 46/87] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Jul 25 12:01:02 2022] GET /identity/v3/projects/23c9d26ac78646c79204b87bd5332e2c => generated 298 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:01:02.206030 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56f2365-52d3-4c3e-9e54-a5b3ae000fe5 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:02.293739 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdcdb91-c94f-49e4-88ce-6722dc9ff222 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:02.337135 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56f2365-52d3-4c3e-9e54-a5b3ae000fe5 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:02.339449 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3162384a-4c8e-49c2-a092-563c351a7e9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:02.340173 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3162384a-4c8e-49c2-a092-563c351a7e9e None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:02.340472 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3162384a-4c8e-49c2-a092-563c351a7e9e None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:02.340758 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3162384a-4c8e-49c2-a092-563c351a7e9e None admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:02.341583 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3162384a-4c8e-49c2-a092-563c351a7e9e None admin] RBAC: Adding query filter params (name=member) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:01:02.341904 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3162384a-4c8e-49c2-a092-563c351a7e9e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:02.343309 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3162384a-4c8e-49c2-a092-563c351a7e9e None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:02.353653 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 42/88] 10.222.0.42 () {56 vars in 1101 bytes} [Mon Jul 25 12:01:02 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:01:02.354581 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3162384a-4c8e-49c2-a092-563c351a7e9e None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:02.356637 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:02.385160 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a4864b-a8e2-4af7-bb7c-df1cfae04a89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:02.385836 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a4864b-a8e2-4af7-bb7c-df1cfae04a89 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:02.386132 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a4864b-a8e2-4af7-bb7c-df1cfae04a89 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:02.386420 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a4864b-a8e2-4af7-bb7c-df1cfae04a89 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:02.387269 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a4864b-a8e2-4af7-bb7c-df1cfae04a89 None admin] RBAC: Adding query filter params (role.id=a7d20885b6d94928a34b3cc5b3a00006, scope.project.id=23c9d26ac78646c79204b87bd5332e2c, user.id=0fad2691baf0472d99015f9f123f8d32) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:01:02.387625 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a4864b-a8e2-4af7-bb7c-df1cfae04a89 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:02.389094 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a4864b-a8e2-4af7-bb7c-df1cfae04a89 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:02.399585 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 47/89] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Jul 25 12:01:02 2022] GET /identity/v3/role_assignments?user.id=0fad2691baf0472d99015f9f123f8d32&scope.project.id=23c9d26ac78646c79204b87bd5332e2c&role.id=a7d20885b6d94928a34b3cc5b3a00006 => generated 259 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:01:02.399585 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a4864b-a8e2-4af7-bb7c-df1cfae04a89 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:02.460251 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:02.540662 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3162384a-4c8e-49c2-a092-563c351a7e9e None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:02.584802 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a4864b-a8e2-4af7-bb7c-df1cfae04a89 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:02.595864 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7221a035-4467-48eb-ac0d-030188b36848 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:02.596554 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7221a035-4467-48eb-ac0d-030188b36848 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:02.596848 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7221a035-4467-48eb-ac0d-030188b36848 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:02.597135 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7221a035-4467-48eb-ac0d-030188b36848 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:02.647300 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc843a8c-69e7-4187-a9dd-033403f2ab31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:02.648058 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc843a8c-69e7-4187-a9dd-033403f2ab31 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:02.648355 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc843a8c-69e7-4187-a9dd-033403f2ab31 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:02.648643 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc843a8c-69e7-4187-a9dd-033403f2ab31 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:02.649778 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc843a8c-69e7-4187-a9dd-033403f2ab31 None admin] RBAC: Adding query filter params (role.id=472313ae69174578824df0bed6390744, scope.project.id=23c9d26ac78646c79204b87bd5332e2c, user.id=b66651fed8eb43758d5d02472aa1ffaf) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:01:02.651030 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc843a8c-69e7-4187-a9dd-033403f2ab31 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:02.653871 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc843a8c-69e7-4187-a9dd-033403f2ab31 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:02.668516 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 48/90] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Jul 25 12:01:02 2022] GET /identity/v3/role_assignments?user.id=b66651fed8eb43758d5d02472aa1ffaf&scope.project.id=23c9d26ac78646c79204b87bd5332e2c&role.id=472313ae69174578824df0bed6390744 => generated 259 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:01:02.670303 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc843a8c-69e7-4187-a9dd-033403f2ab31 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:02.716870 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7221a035-4467-48eb-ac0d-030188b36848 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:02.719429 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7221a035-4467-48eb-ac0d-030188b36848 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:02.754811 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:02.833390 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:02.876283 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc843a8c-69e7-4187-a9dd-033403f2ab31 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:02.912087 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 43/91] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Jul 25 12:01:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 558 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 12:01:02.917489 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7221a035-4467-48eb-ac0d-030188b36848 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:02.960347 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f6befe-079b-4e66-870f-be8c3777c05a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:02.961279 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f6befe-079b-4e66-870f-be8c3777c05a None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:02.961576 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f6befe-079b-4e66-870f-be8c3777c05a None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:02.961865 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f6befe-079b-4e66-870f-be8c3777c05a None admin] PATH_INFO: `/v3/users/b66651fed8eb43758d5d02472aa1ffaf` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:02.963221 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f6befe-079b-4e66-870f-be8c3777c05a None admin] RBAC: Authorizing `identity:get_user(user_id=b66651fed8eb43758d5d02472aa1ffaf)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:02.964755 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f6befe-079b-4e66-870f-be8c3777c05a None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:02.966611 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 49/92] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Jul 25 12:01:02 2022] GET /identity/v3/users/b66651fed8eb43758d5d02472aa1ffaf => generated 275 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:01:02.972358 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f6befe-079b-4e66-870f-be8c3777c05a None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:03.037443 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7221a035-4467-48eb-ac0d-030188b36848 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:03.074936 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16364d7a-98a7-40f8-893c-d57a01d49c82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:03.076171 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16364d7a-98a7-40f8-893c-d57a01d49c82 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:03.076658 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16364d7a-98a7-40f8-893c-d57a01d49c82 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:03.077152 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16364d7a-98a7-40f8-893c-d57a01d49c82 None admin] PATH_INFO: `/v3/projects/23c9d26ac78646c79204b87bd5332e2c` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:03.078587 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16364d7a-98a7-40f8-893c-d57a01d49c82 None admin] RBAC: Authorizing `identity:get_project(project_id=23c9d26ac78646c79204b87bd5332e2c)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:03.080849 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16364d7a-98a7-40f8-893c-d57a01d49c82 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:03.083003 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 44/93] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Jul 25 12:01:02 2022] GET /identity/v3/projects/23c9d26ac78646c79204b87bd5332e2c => generated 298 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:01:03.089553 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16364d7a-98a7-40f8-893c-d57a01d49c82 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:03.160007 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:03.207454 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f6befe-079b-4e66-870f-be8c3777c05a None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:03.226817 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:03.267537 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49afba1-04c7-4314-9eb2-08104ee00ded None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:03.268281 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49afba1-04c7-4314-9eb2-08104ee00ded None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:03.268586 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49afba1-04c7-4314-9eb2-08104ee00ded None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:03.268881 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49afba1-04c7-4314-9eb2-08104ee00ded None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:03.273527 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16364d7a-98a7-40f8-893c-d57a01d49c82 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:03.345719 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6753e1-59b2-4c0b-9c5d-d14ecca041b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:03.346742 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6753e1-59b2-4c0b-9c5d-d14ecca041b1 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:03.347232 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6753e1-59b2-4c0b-9c5d-d14ecca041b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:03.351805 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6753e1-59b2-4c0b-9c5d-d14ecca041b1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:03.352243 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49afba1-04c7-4314-9eb2-08104ee00ded None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:03.353116 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6753e1-59b2-4c0b-9c5d-d14ecca041b1 None admin] RBAC: Adding query filter params (role.id=dea3ef72822043f3baf3bcc1be7c318f, scope.project.id=23c9d26ac78646c79204b87bd5332e2c, user.id=b66651fed8eb43758d5d02472aa1ffaf) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:01:03.353637 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6753e1-59b2-4c0b-9c5d-d14ecca041b1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:03.353736 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49afba1-04c7-4314-9eb2-08104ee00ded None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:03.355367 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6753e1-59b2-4c0b-9c5d-d14ecca041b1 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:03.362295 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 45/94] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Jul 25 12:01:03 2022] GET /identity/v3/role_assignments?user.id=b66651fed8eb43758d5d02472aa1ffaf&scope.project.id=23c9d26ac78646c79204b87bd5332e2c&role.id=dea3ef72822043f3baf3bcc1be7c318f => generated 259 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:01:03.463214 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 50/95] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Jul 25 12:01:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 491 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 25 12:01:03.472920 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6753e1-59b2-4c0b-9c5d-d14ecca041b1 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:03.613233 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:03.618538 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6753e1-59b2-4c0b-9c5d-d14ecca041b1 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:03.660109 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4b2432-6044-40c8-9465-3aea90bf8e71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:03.660989 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4b2432-6044-40c8-9465-3aea90bf8e71 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:03.661280 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4b2432-6044-40c8-9465-3aea90bf8e71 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:03.661565 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4b2432-6044-40c8-9465-3aea90bf8e71 None admin] PATH_INFO: `/v3/projects/946e20d010884b9f84dea264e0a522ea` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:03.662804 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4b2432-6044-40c8-9465-3aea90bf8e71 None admin] RBAC: Authorizing `identity:get_project(project_id=946e20d010884b9f84dea264e0a522ea)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:03.664315 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4b2432-6044-40c8-9465-3aea90bf8e71 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:03.666345 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 46/96] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Jul 25 12:01:03 2022] GET /identity/v3/projects/946e20d010884b9f84dea264e0a522ea => generated 312 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:01:03.672889 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49afba1-04c7-4314-9eb2-08104ee00ded None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:03.684005 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:03.800396 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49afba1-04c7-4314-9eb2-08104ee00ded None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:03.835889 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24cb8ee-212e-406d-b63b-a14040fce726 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:03.836562 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24cb8ee-212e-406d-b63b-a14040fce726 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:03.836852 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24cb8ee-212e-406d-b63b-a14040fce726 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:03.837133 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24cb8ee-212e-406d-b63b-a14040fce726 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:03.837980 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24cb8ee-212e-406d-b63b-a14040fce726 None admin] RBAC: Adding query filter params (role.id=472313ae69174578824df0bed6390744, scope.project.id=946e20d010884b9f84dea264e0a522ea, user.id=b66651fed8eb43758d5d02472aa1ffaf) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:01:03.838356 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24cb8ee-212e-406d-b63b-a14040fce726 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:03.839884 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24cb8ee-212e-406d-b63b-a14040fce726 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:03.845559 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 51/97] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Jul 25 12:01:03 2022] GET /identity/v3/role_assignments?user.id=b66651fed8eb43758d5d02472aa1ffaf&scope.project.id=946e20d010884b9f84dea264e0a522ea&role.id=472313ae69174578824df0bed6390744 => generated 259 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:01:04.933753 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e457ecaf-ad27-4d37-8c43-63e6ff088fc4 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:04.934244 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e457ecaf-ad27-4d37-8c43-63e6ff088fc4 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:04.934315 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e457ecaf-ad27-4d37-8c43-63e6ff088fc4 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:04.935749 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 47/98] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 25 12:01:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:01:04.946560 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c7f1a2-83a0-4ebc-9ef5-06b11da95f13 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:04.947446 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c7f1a2-83a0-4ebc-9ef5-06b11da95f13 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:04.947989 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c7f1a2-83a0-4ebc-9ef5-06b11da95f13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:04.975011 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-c8c7f1a2-83a0-4ebc-9ef5-06b11da95f13 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:01:04.976629 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-c8c7f1a2-83a0-4ebc-9ef5-06b11da95f13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:01:05.059721 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:05.065905 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 52/99] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 25 12:01:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 12:01:05.094495 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee2b6c2-1bc6-4b7c-92a8-9cbc49ddb2e9 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:05.094884 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee2b6c2-1bc6-4b7c-92a8-9cbc49ddb2e9 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:05.095277 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee2b6c2-1bc6-4b7c-92a8-9cbc49ddb2e9 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:05.096770 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 48/100] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 25 12:01:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:01:05.101972 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620f0e61-9e0c-420f-b42e-0028876a215d None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:05.102466 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620f0e61-9e0c-420f-b42e-0028876a215d None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:05.102964 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620f0e61-9e0c-420f-b42e-0028876a215d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:05.105726 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee2b6c2-1bc6-4b7c-92a8-9cbc49ddb2e9 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:05.125978 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-620f0e61-9e0c-420f-b42e-0028876a215d None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:01:05.127746 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-620f0e61-9e0c-420f-b42e-0028876a215d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:01:05.281365 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 53/101] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 25 12:01:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 12:01:05.283347 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d992e5-4bf6-4e2a-b828-65dfd4a28f64 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:05.283669 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d992e5-4bf6-4e2a-b828-65dfd4a28f64 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:05.283957 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d992e5-4bf6-4e2a-b828-65dfd4a28f64 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:05.285241 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 54/102] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 25 12:01:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:01:05.291353 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17036a5-adb9-405d-a173-ef8776b0b543 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:05.291682 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17036a5-adb9-405d-a173-ef8776b0b543 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:05.291969 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17036a5-adb9-405d-a173-ef8776b0b543 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:05.294333 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee2b6c2-1bc6-4b7c-92a8-9cbc49ddb2e9 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:05.321091 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-b17036a5-adb9-405d-a173-ef8776b0b543 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:01:05.323338 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-b17036a5-adb9-405d-a173-ef8776b0b543 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:01:05.365795 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdf2d60-de12-4a38-b411-30ee02b8c6aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:05.366614 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdf2d60-de12-4a38-b411-30ee02b8c6aa None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:05.366908 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdf2d60-de12-4a38-b411-30ee02b8c6aa None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:05.367195 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdf2d60-de12-4a38-b411-30ee02b8c6aa None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:05.384519 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbdf2d60-de12-4a38-b411-30ee02b8c6aa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 12:01:05.392831 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-cbdf2d60-de12-4a38-b411-30ee02b8c6aa None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 25 12:01:05.392831 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:01:05.392831 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:01:05.392831 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:01:05.392831 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:01:05.392831 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:01:05.392831 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:01:05.392831 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:01:05.392831 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:01:05.392831 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:01:05.392831 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:01:05.392831 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:01:05.392831 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 12:01:05.392831 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 12:01:05.392831 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 12:01:05.392831 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 12:01:05.393656 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 12:01:05.393656 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 12:01:05.393656 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 12:01:05.393656 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 12:01:05.393656 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:01:05.393656 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:01:05.393656 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:01:05.393656 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:01:05.393656 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:01:05.393656 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:01:05.393656 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:01:05.393656 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jul 25 12:01:05.393656 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:01:05.393656 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:01:05.393656 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:01:05.393656 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:01:05.394450 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:01:05.394450 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:01:05.394450 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:01:05.394450 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:01:05.394450 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 12:01:05.394450 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 12:01:05.394450 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 12:01:05.394450 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 12:01:05.394450 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 12:01:05.394450 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 12:01:05.394450 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 25 12:01:05.394450 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jul 25 12:01:05.394450 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 49/103] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Jul 25 12:01:05 2022] GET /identity/v3/roles/reader => generated 84 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:01:05.395077 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdf2d60-de12-4a38-b411-30ee02b8c6aa None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:05.422580 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:05.461858 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 55/104] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 25 12:01:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 12:01:05.462712 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17036a5-adb9-405d-a173-ef8776b0b543 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:05.576079 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdf2d60-de12-4a38-b411-30ee02b8c6aa None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:05.577346 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:05.640857 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d5ecee-e65e-4804-9d76-92cc345f6bf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:05.642499 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d5ecee-e65e-4804-9d76-92cc345f6bf7 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:05.642798 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d5ecee-e65e-4804-9d76-92cc345f6bf7 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:05.643090 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d5ecee-e65e-4804-9d76-92cc345f6bf7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:05.644910 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d5ecee-e65e-4804-9d76-92cc345f6bf7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:05.646369 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d5ecee-e65e-4804-9d76-92cc345f6bf7 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:05.649603 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 50/105] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jul 25 12:01:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:01:05.656528 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d5ecee-e65e-4804-9d76-92cc345f6bf7 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:05.664388 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17036a5-adb9-405d-a173-ef8776b0b543 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:05.728235 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e86bbf1-0214-484d-ab43-8f90e7037ebb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:05.728929 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e86bbf1-0214-484d-ab43-8f90e7037ebb None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:05.729223 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e86bbf1-0214-484d-ab43-8f90e7037ebb None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:05.729512 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e86bbf1-0214-484d-ab43-8f90e7037ebb None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:05.730282 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e86bbf1-0214-484d-ab43-8f90e7037ebb None admin] RBAC: Adding query filter params (name=reader) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:01:05.730597 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e86bbf1-0214-484d-ab43-8f90e7037ebb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:05.732043 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e86bbf1-0214-484d-ab43-8f90e7037ebb None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:05.747885 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 56/106] 10.222.0.42 () {56 vars in 1101 bytes} [Mon Jul 25 12:01:05 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:01:05.748757 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e86bbf1-0214-484d-ab43-8f90e7037ebb None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:05.793189 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:05.807779 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d5ecee-e65e-4804-9d76-92cc345f6bf7 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:05.844221 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a3d880-a3e3-437f-a9bd-c2e2a66ee900 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:05.844949 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a3d880-a3e3-437f-a9bd-c2e2a66ee900 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:05.845279 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a3d880-a3e3-437f-a9bd-c2e2a66ee900 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:05.845570 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a3d880-a3e3-437f-a9bd-c2e2a66ee900 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:05.856490 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a3d880-a3e3-437f-a9bd-c2e2a66ee900 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 12:01:05.864487 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-66a3d880-a3e3-437f-a9bd-c2e2a66ee900 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 12:01:05.864487 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:01:05.864487 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:01:05.864487 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:01:05.864487 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:01:05.864487 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:01:05.864487 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:01:05.864487 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:01:05.864487 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:01:05.864487 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:01:05.864487 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:01:05.864487 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:01:05.864487 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 12:01:05.864487 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 12:01:05.864487 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 12:01:05.864487 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 12:01:05.865332 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 12:01:05.865332 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 12:01:05.865332 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 12:01:05.865332 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 12:01:05.865332 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:01:05.865332 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:01:05.865332 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:01:05.865332 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:01:05.865332 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:01:05.865332 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:01:05.865332 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:01:05.865332 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jul 25 12:01:05.865332 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:01:05.865332 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:01:05.865332 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:01:05.865332 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:01:05.866112 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:01:05.866112 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:01:05.866112 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:01:05.866112 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:01:05.866112 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 12:01:05.866112 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 12:01:05.866112 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 12:01:05.866112 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 12:01:05.866112 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 12:01:05.866112 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 12:01:05.866112 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 12:01:05.866112 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jul 25 12:01:05.866112 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 51/107] 10.222.0.42 () {56 vars in 1096 bytes} [Mon Jul 25 12:01:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:01:05.866751 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e86bbf1-0214-484d-ab43-8f90e7037ebb None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:05.867286 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4031675c-b8e8-48da-b263-daf02ecdfc1e None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:05.867449 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4031675c-b8e8-48da-b263-daf02ecdfc1e None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:05.869235 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4031675c-b8e8-48da-b263-daf02ecdfc1e None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:05.869235 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 52/108] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 25 12:01:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:01:05.870167 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4031675c-b8e8-48da-b263-daf02ecdfc1e None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:05.899743 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:05.905270 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b653b7c7-d892-4636-afbe-a0faac6ed3e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:05.905958 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b653b7c7-d892-4636-afbe-a0faac6ed3e1 None admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:05.906252 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b653b7c7-d892-4636-afbe-a0faac6ed3e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:05.906542 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b653b7c7-d892-4636-afbe-a0faac6ed3e1 None admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:05.907267 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b653b7c7-d892-4636-afbe-a0faac6ed3e1 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:05.908801 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b653b7c7-d892-4636-afbe-a0faac6ed3e1 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:05.942837 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 57/109] 10.222.0.42 () {60 vars in 1141 bytes} [Mon Jul 25 12:01:05 2022] POST /identity/v3/projects => generated 302 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:01:05.944614 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df17ba7a-a805-4204-81b0-d09fe5eb47bc None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:05.944976 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df17ba7a-a805-4204-81b0-d09fe5eb47bc None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:05.945269 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df17ba7a-a805-4204-81b0-d09fe5eb47bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:05.968779 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-df17ba7a-a805-4204-81b0-d09fe5eb47bc None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:01:05.970048 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-df17ba7a-a805-4204-81b0-d09fe5eb47bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:01:06.044806 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4031675c-b8e8-48da-b263-daf02ecdfc1e None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:06.059203 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 58/110] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 25 12:01:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 12:01:06.060290 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df17ba7a-a805-4204-81b0-d09fe5eb47bc None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:06.147315 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd64b9e-2a18-4a36-be8f-d8df2ed3fffa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:06.148194 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd64b9e-2a18-4a36-be8f-d8df2ed3fffa None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:06.148490 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd64b9e-2a18-4a36-be8f-d8df2ed3fffa None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:06.148780 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd64b9e-2a18-4a36-be8f-d8df2ed3fffa None admin] PATH_INFO: `/v3/users/77064966b9d74fceaa268d13feebb7c1` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:06.181773 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd64b9e-2a18-4a36-be8f-d8df2ed3fffa None admin] RBAC: Authorizing `identity:get_user(user_id=77064966b9d74fceaa268d13feebb7c1)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:06.183340 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd64b9e-2a18-4a36-be8f-d8df2ed3fffa None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:06.185377 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df17ba7a-a805-4204-81b0-d09fe5eb47bc None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:06.186086 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 53/111] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Jul 25 12:01:05 2022] GET /identity/v3/users/77064966b9d74fceaa268d13feebb7c1 => generated 289 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:01:06.186760 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd64b9e-2a18-4a36-be8f-d8df2ed3fffa None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:06.187950 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:06.199804 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:06.236596 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27329d95-feef-4416-a9fd-6cfefd5a82a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:06.237283 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27329d95-feef-4416-a9fd-6cfefd5a82a0 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:06.237576 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27329d95-feef-4416-a9fd-6cfefd5a82a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:06.238094 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27329d95-feef-4416-a9fd-6cfefd5a82a0 None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:06.239800 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27329d95-feef-4416-a9fd-6cfefd5a82a0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:01:06.240126 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27329d95-feef-4416-a9fd-6cfefd5a82a0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:06.241540 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27329d95-feef-4416-a9fd-6cfefd5a82a0 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:06.252945 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 59/112] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Jul 25 12:01:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:01:06.254444 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27329d95-feef-4416-a9fd-6cfefd5a82a0 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:06.326372 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd64b9e-2a18-4a36-be8f-d8df2ed3fffa None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:06.366309 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0ce325-12fd-4ebd-8478-e0068eb868b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:06.367124 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0ce325-12fd-4ebd-8478-e0068eb868b3 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:06.367418 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0ce325-12fd-4ebd-8478-e0068eb868b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:06.367731 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0ce325-12fd-4ebd-8478-e0068eb868b3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:06.379652 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0ce325-12fd-4ebd-8478-e0068eb868b3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 12:01:06.387657 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-ac0ce325-12fd-4ebd-8478-e0068eb868b3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 25 12:01:06.387657 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:01:06.387657 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:01:06.387657 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:01:06.387657 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:01:06.387657 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:01:06.387657 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:01:06.387657 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:01:06.387657 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:01:06.387657 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:01:06.387657 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:01:06.387657 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:01:06.387657 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 12:01:06.387657 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 12:01:06.387657 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 12:01:06.387657 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 12:01:06.388522 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 12:01:06.388522 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 12:01:06.388522 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 12:01:06.388522 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 12:01:06.388522 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:01:06.388522 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:01:06.388522 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:01:06.388522 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:01:06.388522 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:01:06.388522 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:01:06.388522 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:01:06.388522 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jul 25 12:01:06.388522 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:01:06.388522 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:01:06.388522 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:01:06.388522 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:01:06.389283 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:01:06.389283 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:01:06.389283 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:01:06.389283 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:01:06.389283 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 12:01:06.389283 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 12:01:06.389283 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 12:01:06.389283 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 12:01:06.389283 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 12:01:06.389283 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 12:01:06.389283 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 25 12:01:06.389283 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jul 25 12:01:06.389283 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 54/113] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Jul 25 12:01:06 2022] GET /identity/v3/roles/member => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:01:06.389912 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0ce325-12fd-4ebd-8478-e0068eb868b3 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:06.401800 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27329d95-feef-4416-a9fd-6cfefd5a82a0 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:06.451093 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c24e186-f9fb-4efa-814c-c0108c79d2fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:06.452693 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c24e186-f9fb-4efa-814c-c0108c79d2fc None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:06.453201 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c24e186-f9fb-4efa-814c-c0108c79d2fc None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:06.453687 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c24e186-f9fb-4efa-814c-c0108c79d2fc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:06.463272 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:06.491593 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:06.511624 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0ce325-12fd-4ebd-8478-e0068eb868b3 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:06.531602 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c24e186-f9fb-4efa-814c-c0108c79d2fc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:06.533228 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c24e186-f9fb-4efa-814c-c0108c79d2fc None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:06.556072 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d7bd1e-a83e-4c13-b1f8-5a36e983d5a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:06.557497 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d7bd1e-a83e-4c13-b1f8-5a36e983d5a5 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:06.557996 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d7bd1e-a83e-4c13-b1f8-5a36e983d5a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:06.558714 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d7bd1e-a83e-4c13-b1f8-5a36e983d5a5 None admin] PATH_INFO: `/v3/users/0fad2691baf0472d99015f9f123f8d32` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:06.560854 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d7bd1e-a83e-4c13-b1f8-5a36e983d5a5 None admin] RBAC: Authorizing `identity:get_user(user_id=0fad2691baf0472d99015f9f123f8d32)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:06.563492 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d7bd1e-a83e-4c13-b1f8-5a36e983d5a5 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:06.566603 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 55/114] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Jul 25 12:01:06 2022] GET /identity/v3/users/0fad2691baf0472d99015f9f123f8d32 => generated 247 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:01:06.568048 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c396b2-8f77-454e-a199-9dddfe32b5f9 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:06.568507 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c396b2-8f77-454e-a199-9dddfe32b5f9 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:06.568952 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c396b2-8f77-454e-a199-9dddfe32b5f9 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:06.574585 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 56/115] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 25 12:01:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:01:06.575739 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c396b2-8f77-454e-a199-9dddfe32b5f9 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:06.659904 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 60/116] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Jul 25 12:01:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 12:01:06.661298 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c24e186-f9fb-4efa-814c-c0108c79d2fc None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:06.715314 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c396b2-8f77-454e-a199-9dddfe32b5f9 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:06.734278 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:06.755038 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb49e734-02c7-4505-a222-6757ec86e272 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:06.755752 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb49e734-02c7-4505-a222-6757ec86e272 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:06.756048 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb49e734-02c7-4505-a222-6757ec86e272 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:06.756337 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb49e734-02c7-4505-a222-6757ec86e272 None admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:06.757100 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb49e734-02c7-4505-a222-6757ec86e272 None admin] RBAC: Adding query filter params (name=member) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:01:06.757423 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb49e734-02c7-4505-a222-6757ec86e272 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:06.758827 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb49e734-02c7-4505-a222-6757ec86e272 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:06.760281 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:06.773584 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 57/117] 10.222.0.42 () {56 vars in 1101 bytes} [Mon Jul 25 12:01:06 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:01:06.773584 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0ddf93-4afe-437d-94ba-2afe03034a46 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:06.773584 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0ddf93-4afe-437d-94ba-2afe03034a46 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:06.773584 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0ddf93-4afe-437d-94ba-2afe03034a46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:06.778024 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c24e186-f9fb-4efa-814c-c0108c79d2fc None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:06.790709 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-fc0ddf93-4afe-437d-94ba-2afe03034a46 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:01:06.791954 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-fc0ddf93-4afe-437d-94ba-2afe03034a46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:01:06.820049 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66be774d-b178-4fea-84f4-e2f8d55ee564 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:06.820921 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66be774d-b178-4fea-84f4-e2f8d55ee564 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:06.821216 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66be774d-b178-4fea-84f4-e2f8d55ee564 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:06.821502 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66be774d-b178-4fea-84f4-e2f8d55ee564 None admin] PATH_INFO: `/v3/projects/23c9d26ac78646c79204b87bd5332e2c` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:06.822717 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66be774d-b178-4fea-84f4-e2f8d55ee564 None admin] RBAC: Authorizing `identity:get_project(project_id=23c9d26ac78646c79204b87bd5332e2c)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:06.824261 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66be774d-b178-4fea-84f4-e2f8d55ee564 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:06.825976 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 61/118] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Jul 25 12:01:06 2022] GET /identity/v3/projects/23c9d26ac78646c79204b87bd5332e2c => generated 298 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:01:06.831398 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66be774d-b178-4fea-84f4-e2f8d55ee564 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:06.866819 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 58/119] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 25 12:01:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 12:01:06.868182 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa176cb-6754-4feb-9606-50182311e2e6 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:06.868486 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa176cb-6754-4feb-9606-50182311e2e6 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:06.868778 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa176cb-6754-4feb-9606-50182311e2e6 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:06.871581 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 59/120] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 25 12:01:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:01:06.871581 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa176cb-6754-4feb-9606-50182311e2e6 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:06.947619 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66be774d-b178-4fea-84f4-e2f8d55ee564 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:06.983505 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dadba63-c825-4744-b7eb-b61a335ee947 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:06.984488 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dadba63-c825-4744-b7eb-b61a335ee947 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:06.984873 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dadba63-c825-4744-b7eb-b61a335ee947 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:06.985166 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dadba63-c825-4744-b7eb-b61a335ee947 None admin] PATH_INFO: `/v3/projects/23c9d26ac78646c79204b87bd5332e2c` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:06.986889 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dadba63-c825-4744-b7eb-b61a335ee947 None admin] RBAC: Authorizing `identity:get_project(project_id=23c9d26ac78646c79204b87bd5332e2c)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:06.988369 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dadba63-c825-4744-b7eb-b61a335ee947 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:06.989458 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa176cb-6754-4feb-9606-50182311e2e6 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:06.990075 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 62/121] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Jul 25 12:01:06 2022] GET /identity/v3/projects/23c9d26ac78646c79204b87bd5332e2c => generated 298 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:01:06.995564 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dadba63-c825-4744-b7eb-b61a335ee947 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:06.998266 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:07.017741 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:07.027358 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1596410f-621f-476b-a98c-da465f60bd6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:07.028486 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1596410f-621f-476b-a98c-da465f60bd6c None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:07.028781 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1596410f-621f-476b-a98c-da465f60bd6c None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:07.029066 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1596410f-621f-476b-a98c-da465f60bd6c None admin] PATH_INFO: `/v3/users/b66651fed8eb43758d5d02472aa1ffaf` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:07.030269 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1596410f-621f-476b-a98c-da465f60bd6c None admin] RBAC: Authorizing `identity:get_user(user_id=b66651fed8eb43758d5d02472aa1ffaf)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:07.031955 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1596410f-621f-476b-a98c-da465f60bd6c None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:07.033903 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 60/122] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Jul 25 12:01:06 2022] GET /identity/v3/users/b66651fed8eb43758d5d02472aa1ffaf => generated 275 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:01:07.039229 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2f876e-5906-4e45-9b65-fb5781bf62e4 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:07.039528 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2f876e-5906-4e45-9b65-fb5781bf62e4 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:07.040094 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2f876e-5906-4e45-9b65-fb5781bf62e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:07.057774 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-7f2f876e-5906-4e45-9b65-fb5781bf62e4 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:01:07.059000 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-7f2f876e-5906-4e45-9b65-fb5781bf62e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:01:07.099770 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dadba63-c825-4744-b7eb-b61a335ee947 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:07.135043 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 61/123] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 25 12:01:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 12:01:07.136181 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2f876e-5906-4e45-9b65-fb5781bf62e4 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:07.140457 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4225382b-d23d-45a0-85c9-239f28348583 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:07.141362 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4225382b-d23d-45a0-85c9-239f28348583 None admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:07.141647 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4225382b-d23d-45a0-85c9-239f28348583 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:07.141931 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4225382b-d23d-45a0-85c9-239f28348583 None admin] PATH_INFO: `/v3/projects/23c9d26ac78646c79204b87bd5332e2c/users/0fad2691baf0472d99015f9f123f8d32/roles/a7d20885b6d94928a34b3cc5b3a00006` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:07.144964 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4225382b-d23d-45a0-85c9-239f28348583 None admin] RBAC: Authorizing `identity:create_grant(project_id=23c9d26ac78646c79204b87bd5332e2c, user_id=0fad2691baf0472d99015f9f123f8d32, role_id=a7d20885b6d94928a34b3cc5b3a00006)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:07.146786 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4225382b-d23d-45a0-85c9-239f28348583 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:07.172603 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 63/124] 10.222.0.42 () {58 vars in 1439 bytes} [Mon Jul 25 12:01:06 2022] PUT /identity/v3/projects/23c9d26ac78646c79204b87bd5332e2c/users/0fad2691baf0472d99015f9f123f8d32/roles/a7d20885b6d94928a34b3cc5b3a00006 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:01:07.172603 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4225382b-d23d-45a0-85c9-239f28348583 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:07.248087 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:07.269438 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2f876e-5906-4e45-9b65-fb5781bf62e4 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:07.303231 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4225382b-d23d-45a0-85c9-239f28348583 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:07.304109 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:07.319464 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d9e258-4167-4cd7-9d99-0238a09cf5ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:07.320431 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d9e258-4167-4cd7-9d99-0238a09cf5ab None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:07.320794 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d9e258-4167-4cd7-9d99-0238a09cf5ab None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:07.321207 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d9e258-4167-4cd7-9d99-0238a09cf5ab None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:07.337351 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d9e258-4167-4cd7-9d99-0238a09cf5ab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 12:01:07.347240 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-28d9e258-4167-4cd7-9d99-0238a09cf5ab None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 12:01:07.347240 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:01:07.347240 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:01:07.347240 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:01:07.347240 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:01:07.347240 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:01:07.347240 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:01:07.347240 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:01:07.347240 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:01:07.347240 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:01:07.347240 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:01:07.347240 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:01:07.347240 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 12:01:07.347240 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 12:01:07.347240 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 12:01:07.347240 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 12:01:07.348243 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 12:01:07.348243 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 12:01:07.348243 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 12:01:07.348243 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 12:01:07.348243 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:01:07.348243 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:01:07.348243 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:01:07.348243 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:01:07.348243 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:01:07.348243 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:01:07.348243 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:01:07.348243 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jul 25 12:01:07.348243 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:01:07.348243 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:01:07.348243 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:01:07.348243 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:01:07.349012 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:01:07.349012 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:01:07.349012 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:01:07.349012 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:01:07.349012 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 12:01:07.349012 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 12:01:07.349012 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 12:01:07.349012 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 12:01:07.349012 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 12:01:07.349012 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 12:01:07.349012 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 12:01:07.349012 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jul 25 12:01:07.349012 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 62/125] 10.222.0.42 () {56 vars in 1114 bytes} [Mon Jul 25 12:01:07 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:01:07.349622 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d9e258-4167-4cd7-9d99-0238a09cf5ab None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:07.363795 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f6f1b4-2ebe-48d9-97d6-af2512054ec7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:07.364574 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f6f1b4-2ebe-48d9-97d6-af2512054ec7 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:07.364937 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f6f1b4-2ebe-48d9-97d6-af2512054ec7 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:07.366529 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f6f1b4-2ebe-48d9-97d6-af2512054ec7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:07.366529 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f6f1b4-2ebe-48d9-97d6-af2512054ec7 None admin] RBAC: Adding query filter params (role.id=716f62e98b0c4df69e1441697434334c, scope.project.id=23c9d26ac78646c79204b87bd5332e2c, user.id=77064966b9d74fceaa268d13feebb7c1) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:01:07.366781 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f6f1b4-2ebe-48d9-97d6-af2512054ec7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:07.368469 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f6f1b4-2ebe-48d9-97d6-af2512054ec7 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:07.377110 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 64/126] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Jul 25 12:01:07 2022] GET /identity/v3/role_assignments?user.id=77064966b9d74fceaa268d13feebb7c1&scope.project.id=23c9d26ac78646c79204b87bd5332e2c&role.id=716f62e98b0c4df69e1441697434334c => generated 259 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:01:07.378039 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f6f1b4-2ebe-48d9-97d6-af2512054ec7 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:07.490105 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d9e258-4167-4cd7-9d99-0238a09cf5ab None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:07.515941 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f6f1b4-2ebe-48d9-97d6-af2512054ec7 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:07.544108 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005784bc-1f5b-4910-9f95-9bb1eba81d1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:07.545336 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005784bc-1f5b-4910-9f95-9bb1eba81d1d None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:07.545643 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005784bc-1f5b-4910-9f95-9bb1eba81d1d None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:07.545932 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005784bc-1f5b-4910-9f95-9bb1eba81d1d None admin] PATH_INFO: `/v3/projects/23c9d26ac78646c79204b87bd5332e2c` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:07.547336 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005784bc-1f5b-4910-9f95-9bb1eba81d1d None admin] RBAC: Authorizing `identity:get_project(project_id=23c9d26ac78646c79204b87bd5332e2c)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:07.549528 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005784bc-1f5b-4910-9f95-9bb1eba81d1d None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:07.553224 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 63/127] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Jul 25 12:01:07 2022] GET /identity/v3/projects/23c9d26ac78646c79204b87bd5332e2c => generated 298 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:01:07.553224 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005784bc-1f5b-4910-9f95-9bb1eba81d1d None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:07.580696 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcda3577-8e58-4519-8673-d400113cf2f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:07.582701 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcda3577-8e58-4519-8673-d400113cf2f8 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:07.583316 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcda3577-8e58-4519-8673-d400113cf2f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:07.588361 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:07.589341 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcda3577-8e58-4519-8673-d400113cf2f8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:07.605110 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcda3577-8e58-4519-8673-d400113cf2f8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 12:01:07.617335 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-bcda3577-8e58-4519-8673-d400113cf2f8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 25 12:01:07.617335 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:01:07.617335 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:01:07.617335 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:01:07.617335 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:01:07.617335 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:01:07.617335 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:01:07.617335 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:01:07.617335 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:01:07.617335 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:01:07.617335 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:01:07.617335 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:01:07.617335 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 12:01:07.617335 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 12:01:07.617335 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 12:01:07.617335 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 12:01:07.618251 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 12:01:07.618251 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 12:01:07.618251 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 12:01:07.618251 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 12:01:07.618251 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:01:07.618251 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:01:07.618251 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:01:07.618251 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:01:07.618251 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:01:07.618251 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:01:07.618251 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:01:07.618251 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 25 12:01:07.618251 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:01:07.618251 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:01:07.618251 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:01:07.618251 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:01:07.619009 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:01:07.619009 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:01:07.619009 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:01:07.619009 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:01:07.619009 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 12:01:07.619009 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 12:01:07.619009 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 12:01:07.619009 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 12:01:07.619009 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 12:01:07.619009 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 12:01:07.619009 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 25 12:01:07.619009 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 25 12:01:07.620860 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:07.623368 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 65/128] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Jul 25 12:01:07 2022] GET /identity/v3/roles/member => generated 84 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:01:07.625021 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcda3577-8e58-4519-8673-d400113cf2f8 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:07.739380 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005784bc-1f5b-4910-9f95-9bb1eba81d1d None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:07.805592 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec89be3-efb0-4422-92d9-40c6e35e1809 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:07.806282 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec89be3-efb0-4422-92d9-40c6e35e1809 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:07.806581 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec89be3-efb0-4422-92d9-40c6e35e1809 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:07.806873 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec89be3-efb0-4422-92d9-40c6e35e1809 None admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:07.807664 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec89be3-efb0-4422-92d9-40c6e35e1809 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:01:07.807988 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec89be3-efb0-4422-92d9-40c6e35e1809 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:07.809385 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec89be3-efb0-4422-92d9-40c6e35e1809 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:07.812639 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcda3577-8e58-4519-8673-d400113cf2f8 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:07.828943 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 64/129] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Jul 25 12:01:07 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:01:07.829856 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec89be3-efb0-4422-92d9-40c6e35e1809 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:07.875645 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbfb9fe-43a9-4d9b-be22-ebb545537893 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:07.877121 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbfb9fe-43a9-4d9b-be22-ebb545537893 None admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:07.877416 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbfb9fe-43a9-4d9b-be22-ebb545537893 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:07.877704 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbfb9fe-43a9-4d9b-be22-ebb545537893 None admin] PATH_INFO: `/v3/projects/23c9d26ac78646c79204b87bd5332e2c/users/b66651fed8eb43758d5d02472aa1ffaf/roles/472313ae69174578824df0bed6390744` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:07.881166 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbfb9fe-43a9-4d9b-be22-ebb545537893 None admin] RBAC: Authorizing `identity:create_grant(project_id=23c9d26ac78646c79204b87bd5332e2c, user_id=b66651fed8eb43758d5d02472aa1ffaf, role_id=472313ae69174578824df0bed6390744)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:07.882640 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbfb9fe-43a9-4d9b-be22-ebb545537893 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:07.911212 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 66/130] 10.222.0.42 () {58 vars in 1439 bytes} [Mon Jul 25 12:01:07 2022] PUT /identity/v3/projects/23c9d26ac78646c79204b87bd5332e2c/users/b66651fed8eb43758d5d02472aa1ffaf/roles/472313ae69174578824df0bed6390744 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:01:07.911983 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbfb9fe-43a9-4d9b-be22-ebb545537893 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:08.007154 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:08.032524 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec89be3-efb0-4422-92d9-40c6e35e1809 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:08.067948 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:08.108001 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344b00d3-7105-49fb-a7b4-c93746557723 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:08.108766 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344b00d3-7105-49fb-a7b4-c93746557723 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:08.109064 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344b00d3-7105-49fb-a7b4-c93746557723 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:08.109354 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344b00d3-7105-49fb-a7b4-c93746557723 None admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:08.110139 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344b00d3-7105-49fb-a7b4-c93746557723 None admin] RBAC: Adding query filter params (name=member) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:01:08.110459 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344b00d3-7105-49fb-a7b4-c93746557723 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:08.112689 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344b00d3-7105-49fb-a7b4-c93746557723 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:08.118701 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbfb9fe-43a9-4d9b-be22-ebb545537893 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:08.131938 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 65/131] 10.222.0.42 () {56 vars in 1101 bytes} [Mon Jul 25 12:01:07 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:01:08.140610 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344b00d3-7105-49fb-a7b4-c93746557723 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:08.196067 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f297fa3-bfbc-4960-8d6b-cea3e1b72c1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:08.197013 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f297fa3-bfbc-4960-8d6b-cea3e1b72c1f None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:08.197387 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f297fa3-bfbc-4960-8d6b-cea3e1b72c1f None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:08.197681 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f297fa3-bfbc-4960-8d6b-cea3e1b72c1f None admin] PATH_INFO: `/v3/users/b66651fed8eb43758d5d02472aa1ffaf` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:08.202780 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f297fa3-bfbc-4960-8d6b-cea3e1b72c1f None admin] RBAC: Authorizing `identity:get_user(user_id=b66651fed8eb43758d5d02472aa1ffaf)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:08.204325 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f297fa3-bfbc-4960-8d6b-cea3e1b72c1f None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:08.206449 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 67/132] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Jul 25 12:01:07 2022] GET /identity/v3/users/b66651fed8eb43758d5d02472aa1ffaf => generated 275 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:01:08.212890 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f297fa3-bfbc-4960-8d6b-cea3e1b72c1f None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:08.331719 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344b00d3-7105-49fb-a7b4-c93746557723 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:08.395900 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03ce36b-3eb8-4c5f-b5ea-41b5cba111aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:08.396802 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03ce36b-3eb8-4c5f-b5ea-41b5cba111aa None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:08.397096 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03ce36b-3eb8-4c5f-b5ea-41b5cba111aa None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:08.397383 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03ce36b-3eb8-4c5f-b5ea-41b5cba111aa None admin] PATH_INFO: `/v3/users/b66651fed8eb43758d5d02472aa1ffaf` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:08.397480 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f297fa3-bfbc-4960-8d6b-cea3e1b72c1f None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:08.398624 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03ce36b-3eb8-4c5f-b5ea-41b5cba111aa None admin] RBAC: Authorizing `identity:get_user(user_id=b66651fed8eb43758d5d02472aa1ffaf)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:08.401278 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03ce36b-3eb8-4c5f-b5ea-41b5cba111aa None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:08.403112 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 66/133] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Jul 25 12:01:08 2022] GET /identity/v3/users/b66651fed8eb43758d5d02472aa1ffaf => generated 275 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:01:08.404173 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3b0fac-774b-43c1-9a7a-43b0fc8cc433 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:08.404468 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3b0fac-774b-43c1-9a7a-43b0fc8cc433 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:08.404758 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3b0fac-774b-43c1-9a7a-43b0fc8cc433 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:08.411917 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 67/134] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 25 12:01:08 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:01:08.412825 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3b0fac-774b-43c1-9a7a-43b0fc8cc433 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:08.433103 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:08.472096 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:08.504123 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc894aea-b20d-482c-be4f-f1fe80c00b28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:08.505021 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc894aea-b20d-482c-be4f-f1fe80c00b28 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:08.505320 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc894aea-b20d-482c-be4f-f1fe80c00b28 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:08.505606 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc894aea-b20d-482c-be4f-f1fe80c00b28 None admin] PATH_INFO: `/v3/projects/23c9d26ac78646c79204b87bd5332e2c` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:08.506863 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc894aea-b20d-482c-be4f-f1fe80c00b28 None admin] RBAC: Authorizing `identity:get_project(project_id=23c9d26ac78646c79204b87bd5332e2c)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:08.508352 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc894aea-b20d-482c-be4f-f1fe80c00b28 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:08.510370 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 68/135] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Jul 25 12:01:08 2022] GET /identity/v3/projects/23c9d26ac78646c79204b87bd5332e2c => generated 298 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 12:01:08.511579 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befec597-5244-4f49-a10d-4db90205c50e None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:08.511856 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befec597-5244-4f49-a10d-4db90205c50e None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:08.512148 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befec597-5244-4f49-a10d-4db90205c50e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:08.538823 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3b0fac-774b-43c1-9a7a-43b0fc8cc433 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:08.555668 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-befec597-5244-4f49-a10d-4db90205c50e None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:01:08.556884 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-befec597-5244-4f49-a10d-4db90205c50e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:01:08.599295 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7192597-154b-40f5-b0f3-2be76675eff8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:08.600040 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7192597-154b-40f5-b0f3-2be76675eff8 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:08.600345 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7192597-154b-40f5-b0f3-2be76675eff8 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:08.600640 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7192597-154b-40f5-b0f3-2be76675eff8 None admin] PATH_INFO: `/v3/projects/946e20d010884b9f84dea264e0a522ea` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:08.602864 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7192597-154b-40f5-b0f3-2be76675eff8 None admin] RBAC: Authorizing `identity:get_project(project_id=946e20d010884b9f84dea264e0a522ea)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:08.604400 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7192597-154b-40f5-b0f3-2be76675eff8 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:08.607656 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 68/136] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Jul 25 12:01:08 2022] GET /identity/v3/projects/946e20d010884b9f84dea264e0a522ea => generated 312 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:01:08.608499 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7192597-154b-40f5-b0f3-2be76675eff8 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:08.694468 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 69/137] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 25 12:01:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 12:01:08.695404 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befec597-5244-4f49-a10d-4db90205c50e None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:08.784710 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7192597-154b-40f5-b0f3-2be76675eff8 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:08.808099 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:08.863887 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce18bdcf-17a0-4fc3-9f8b-39961482b719 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:08.864806 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce18bdcf-17a0-4fc3-9f8b-39961482b719 None admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:08.865103 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce18bdcf-17a0-4fc3-9f8b-39961482b719 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:08.865394 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce18bdcf-17a0-4fc3-9f8b-39961482b719 None admin] PATH_INFO: `/v3/projects/23c9d26ac78646c79204b87bd5332e2c/users/b66651fed8eb43758d5d02472aa1ffaf/roles/dea3ef72822043f3baf3bcc1be7c318f` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:08.866878 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befec597-5244-4f49-a10d-4db90205c50e None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:08.870675 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce18bdcf-17a0-4fc3-9f8b-39961482b719 None admin] RBAC: Authorizing `identity:create_grant(project_id=23c9d26ac78646c79204b87bd5332e2c, user_id=b66651fed8eb43758d5d02472aa1ffaf, role_id=dea3ef72822043f3baf3bcc1be7c318f)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:08.872172 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce18bdcf-17a0-4fc3-9f8b-39961482b719 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:08.883366 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:08.902330 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 69/138] 10.222.0.42 () {58 vars in 1439 bytes} [Mon Jul 25 12:01:08 2022] PUT /identity/v3/projects/23c9d26ac78646c79204b87bd5332e2c/users/b66651fed8eb43758d5d02472aa1ffaf/roles/dea3ef72822043f3baf3bcc1be7c318f => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:01:08.904119 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce18bdcf-17a0-4fc3-9f8b-39961482b719 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:08.931018 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99de0061-8164-4094-8400-918891fd5012 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:08.933241 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99de0061-8164-4094-8400-918891fd5012 None admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:08.933536 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99de0061-8164-4094-8400-918891fd5012 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:08.934681 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99de0061-8164-4094-8400-918891fd5012 None admin] PATH_INFO: `/v3/projects/946e20d010884b9f84dea264e0a522ea/users/b66651fed8eb43758d5d02472aa1ffaf/roles/472313ae69174578824df0bed6390744` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:08.940322 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99de0061-8164-4094-8400-918891fd5012 None admin] RBAC: Authorizing `identity:create_grant(project_id=946e20d010884b9f84dea264e0a522ea, user_id=b66651fed8eb43758d5d02472aa1ffaf, role_id=472313ae69174578824df0bed6390744)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:08.942352 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99de0061-8164-4094-8400-918891fd5012 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:08.968174 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 70/139] 10.222.0.42 () {58 vars in 1439 bytes} [Mon Jul 25 12:01:08 2022] PUT /identity/v3/projects/946e20d010884b9f84dea264e0a522ea/users/b66651fed8eb43758d5d02472aa1ffaf/roles/472313ae69174578824df0bed6390744 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:01:09.058771 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce18bdcf-17a0-4fc3-9f8b-39961482b719 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:09.108843 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe05813-990b-42ea-a34a-dd00abcc0b37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:09.109728 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe05813-990b-42ea-a34a-dd00abcc0b37 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:09.110021 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe05813-990b-42ea-a34a-dd00abcc0b37 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:09.110360 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe05813-990b-42ea-a34a-dd00abcc0b37 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:09.115043 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe05813-990b-42ea-a34a-dd00abcc0b37 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:09.116511 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe05813-990b-42ea-a34a-dd00abcc0b37 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:09.118285 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 70/140] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jul 25 12:01:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:01:09.123938 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99de0061-8164-4094-8400-918891fd5012 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:09.257266 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99de0061-8164-4094-8400-918891fd5012 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:09.298845 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc25ee6-fa9e-45a0-8a3f-de5d2c5638eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:09.299563 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc25ee6-fa9e-45a0-8a3f-de5d2c5638eb None admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:09.299859 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc25ee6-fa9e-45a0-8a3f-de5d2c5638eb None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:09.300149 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc25ee6-fa9e-45a0-8a3f-de5d2c5638eb None admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:09.300884 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc25ee6-fa9e-45a0-8a3f-de5d2c5638eb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:09.302363 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc25ee6-fa9e-45a0-8a3f-de5d2c5638eb None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:09.327330 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 71/141] 10.222.0.42 () {60 vars in 1132 bytes} [Mon Jul 25 12:01:09 2022] POST /identity/v3/users => generated 283 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:01:09.763600 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5b3b83-6f4e-400d-b4f5-f786ba268761 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:09.763935 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5b3b83-6f4e-400d-b4f5-f786ba268761 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:09.764182 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5b3b83-6f4e-400d-b4f5-f786ba268761 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:09.765851 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 71/142] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 25 12:01:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:01:09.779058 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5261dc9-56a6-4930-a4df-6be850cf0b8b None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:09.779058 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5261dc9-56a6-4930-a4df-6be850cf0b8b None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:09.779646 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5261dc9-56a6-4930-a4df-6be850cf0b8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:09.800897 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-f5261dc9-56a6-4930-a4df-6be850cf0b8b None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:01:09.802228 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-f5261dc9-56a6-4930-a4df-6be850cf0b8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:01:09.826522 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:09.879853 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 72/143] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 25 12:01:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 12:01:09.903604 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5b3b83-6f4e-400d-b4f5-f786ba268761 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:09.904703 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e055e5bb-9b94-48f5-b562-59a8d9a460c0 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:09.905204 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e055e5bb-9b94-48f5-b562-59a8d9a460c0 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:09.905699 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e055e5bb-9b94-48f5-b562-59a8d9a460c0 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:09.907780 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 73/144] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 25 12:01:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:01:09.913615 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c34d74a-2de3-4def-b396-dd315256fd53 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:09.914162 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c34d74a-2de3-4def-b396-dd315256fd53 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:09.914667 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c34d74a-2de3-4def-b396-dd315256fd53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:09.946315 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-6c34d74a-2de3-4def-b396-dd315256fd53 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:01:09.947969 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-6c34d74a-2de3-4def-b396-dd315256fd53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:01:09.960239 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:10.025100 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5b3b83-6f4e-400d-b4f5-f786ba268761 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:10.071723 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736cd06f-968a-41cd-a187-7e0ff4c4a404 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:10.072951 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736cd06f-968a-41cd-a187-7e0ff4c4a404 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:10.073250 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736cd06f-968a-41cd-a187-7e0ff4c4a404 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:10.073539 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736cd06f-968a-41cd-a187-7e0ff4c4a404 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:10.086356 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736cd06f-968a-41cd-a187-7e0ff4c4a404 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 12:01:10.094765 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-736cd06f-968a-41cd-a187-7e0ff4c4a404 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 12:01:10.094765 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:01:10.094765 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:01:10.094765 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:01:10.094765 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:01:10.094765 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:01:10.094765 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:01:10.094765 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:01:10.094765 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:01:10.094765 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:01:10.094765 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:01:10.094765 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:01:10.094765 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 12:01:10.094765 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 12:01:10.094765 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 12:01:10.094765 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 12:01:10.096270 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 12:01:10.096270 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 12:01:10.096270 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 12:01:10.096270 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 12:01:10.096270 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:01:10.096270 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:01:10.096270 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:01:10.096270 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:01:10.096270 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:01:10.096270 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:01:10.096270 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:01:10.096270 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jul 25 12:01:10.096270 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:01:10.096270 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:01:10.096270 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:01:10.096270 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:01:10.097046 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:01:10.097046 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:01:10.097046 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:01:10.097046 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:01:10.097046 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 12:01:10.097046 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 12:01:10.097046 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 12:01:10.097046 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 12:01:10.097046 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 12:01:10.097046 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 12:01:10.097046 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 12:01:10.097046 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jul 25 12:01:10.097046 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 72/145] 10.222.0.42 () {56 vars in 1096 bytes} [Mon Jul 25 12:01:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:01:10.103645 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736cd06f-968a-41cd-a187-7e0ff4c4a404 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:10.153265 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 74/146] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 25 12:01:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 25 12:01:10.192330 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c34d74a-2de3-4def-b396-dd315256fd53 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:10.215455 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736cd06f-968a-41cd-a187-7e0ff4c4a404 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:10.219883 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:10.254177 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0069e978-3fe4-4087-a403-3199841ffa82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:10.254874 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0069e978-3fe4-4087-a403-3199841ffa82 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:10.255168 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0069e978-3fe4-4087-a403-3199841ffa82 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:10.255461 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0069e978-3fe4-4087-a403-3199841ffa82 None admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:10.256308 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0069e978-3fe4-4087-a403-3199841ffa82 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:01:10.256638 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0069e978-3fe4-4087-a403-3199841ffa82 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:10.258901 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0069e978-3fe4-4087-a403-3199841ffa82 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:10.270765 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 73/147] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Jul 25 12:01:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 12:01:10.276757 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0069e978-3fe4-4087-a403-3199841ffa82 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:10.398041 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0069e978-3fe4-4087-a403-3199841ffa82 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:10.437729 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca35202-fbf7-4c16-b9f8-a636dc393405 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:10.438629 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca35202-fbf7-4c16-b9f8-a636dc393405 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:10.438951 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca35202-fbf7-4c16-b9f8-a636dc393405 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:10.439242 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca35202-fbf7-4c16-b9f8-a636dc393405 None admin] PATH_INFO: `/v3/users/0fad2691baf0472d99015f9f123f8d32` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:10.440647 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca35202-fbf7-4c16-b9f8-a636dc393405 None admin] RBAC: Authorizing `identity:get_user(user_id=0fad2691baf0472d99015f9f123f8d32)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:10.442102 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca35202-fbf7-4c16-b9f8-a636dc393405 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:10.442598 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:10.445290 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 74/148] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Jul 25 12:01:10 2022] GET /identity/v3/users/0fad2691baf0472d99015f9f123f8d32 => generated 247 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 12:01:10.451139 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca35202-fbf7-4c16-b9f8-a636dc393405 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:10.463334 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c34d74a-2de3-4def-b396-dd315256fd53 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:10.491984 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:10.524256 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab84ecb8-ff68-4f4f-aa68-da6136af26b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:10.525075 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab84ecb8-ff68-4f4f-aa68-da6136af26b4 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:10.525366 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab84ecb8-ff68-4f4f-aa68-da6136af26b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:10.525644 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab84ecb8-ff68-4f4f-aa68-da6136af26b4 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:10.546631 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab84ecb8-ff68-4f4f-aa68-da6136af26b4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 12:01:10.555010 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-ab84ecb8-ff68-4f4f-aa68-da6136af26b4 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 25 12:01:10.555010 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:01:10.555010 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:01:10.555010 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:01:10.555010 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:01:10.555010 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:01:10.555010 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:01:10.555010 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:01:10.555010 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:01:10.555010 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:01:10.555010 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:01:10.555010 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:01:10.555010 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 12:01:10.555010 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 12:01:10.555010 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 12:01:10.555010 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 12:01:10.555895 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 12:01:10.555895 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 12:01:10.555895 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 12:01:10.555895 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 12:01:10.555895 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:01:10.555895 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:01:10.555895 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:01:10.555895 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:01:10.555895 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:01:10.555895 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:01:10.555895 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:01:10.555895 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 25 12:01:10.555895 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:01:10.555895 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:01:10.555895 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:01:10.555895 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:01:10.556656 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:01:10.556656 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:01:10.556656 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:01:10.556656 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:01:10.556656 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 12:01:10.556656 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 12:01:10.556656 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 12:01:10.556656 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 12:01:10.556656 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 12:01:10.556656 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 12:01:10.556656 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 25 12:01:10.556656 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 25 12:01:10.563072 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 75/149] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Jul 25 12:01:10 2022] GET /identity/v3/roles/reader => generated 84 bytes in 371 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:01:10.565235 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca35202-fbf7-4c16-b9f8-a636dc393405 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:10.570254 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab84ecb8-ff68-4f4f-aa68-da6136af26b4 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:10.612661 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be27385-de11-495f-bd04-b820913722dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:10.613329 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be27385-de11-495f-bd04-b820913722dd None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:10.613619 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be27385-de11-495f-bd04-b820913722dd None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:10.613955 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be27385-de11-495f-bd04-b820913722dd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:10.691272 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be27385-de11-495f-bd04-b820913722dd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:10.692791 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be27385-de11-495f-bd04-b820913722dd None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:10.718682 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab84ecb8-ff68-4f4f-aa68-da6136af26b4 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:10.727988 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:10.786436 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a33492-cbea-4d87-b652-f96f18e4b2e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:10.787355 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a33492-cbea-4d87-b652-f96f18e4b2e8 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:10.795985 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a33492-cbea-4d87-b652-f96f18e4b2e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:10.796525 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a33492-cbea-4d87-b652-f96f18e4b2e8 None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:10.797602 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a33492-cbea-4d87-b652-f96f18e4b2e8 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:01:10.798101 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a33492-cbea-4d87-b652-f96f18e4b2e8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:10.799783 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a33492-cbea-4d87-b652-f96f18e4b2e8 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:10.821900 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 76/150] 10.222.0.42 () {56 vars in 1101 bytes} [Mon Jul 25 12:01:10 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:01:10.834583 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 75/151] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Jul 25 12:01:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 12:01:10.835887 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a33492-cbea-4d87-b652-f96f18e4b2e8 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:10.842558 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be27385-de11-495f-bd04-b820913722dd None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:10.863926 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:10.976986 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be27385-de11-495f-bd04-b820913722dd None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:11.020209 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:11.044974 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a006f83-245f-421a-91e8-4a5deaeec849 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:11.045873 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a006f83-245f-421a-91e8-4a5deaeec849 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:11.046170 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a006f83-245f-421a-91e8-4a5deaeec849 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:11.046450 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a006f83-245f-421a-91e8-4a5deaeec849 None admin] PATH_INFO: `/v3/projects/23c9d26ac78646c79204b87bd5332e2c` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:11.052500 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a006f83-245f-421a-91e8-4a5deaeec849 None admin] RBAC: Authorizing `identity:get_project(project_id=23c9d26ac78646c79204b87bd5332e2c)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:11.053938 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a006f83-245f-421a-91e8-4a5deaeec849 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:11.055954 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 76/152] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Jul 25 12:01:10 2022] GET /identity/v3/projects/23c9d26ac78646c79204b87bd5332e2c => generated 298 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 12:01:11.059643 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a33492-cbea-4d87-b652-f96f18e4b2e8 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:11.073786 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a006f83-245f-421a-91e8-4a5deaeec849 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:11.112891 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36caf3fa-c9ec-425e-9fad-a794b1eda884 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:11.113831 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36caf3fa-c9ec-425e-9fad-a794b1eda884 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:11.114120 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36caf3fa-c9ec-425e-9fad-a794b1eda884 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:11.114400 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36caf3fa-c9ec-425e-9fad-a794b1eda884 None admin] PATH_INFO: `/v3/users/77064966b9d74fceaa268d13feebb7c1` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:11.115649 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36caf3fa-c9ec-425e-9fad-a794b1eda884 None admin] RBAC: Authorizing `identity:get_user(user_id=77064966b9d74fceaa268d13feebb7c1)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:11.117183 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36caf3fa-c9ec-425e-9fad-a794b1eda884 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:11.118832 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 77/153] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Jul 25 12:01:10 2022] GET /identity/v3/users/77064966b9d74fceaa268d13feebb7c1 => generated 289 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:01:11.124431 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36caf3fa-c9ec-425e-9fad-a794b1eda884 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:11.214029 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a006f83-245f-421a-91e8-4a5deaeec849 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:11.265256 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0df27a-53e7-469c-b9e5-be948a9e814a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:11.265997 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0df27a-53e7-469c-b9e5-be948a9e814a None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:11.266214 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0df27a-53e7-469c-b9e5-be948a9e814a None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:11.266500 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0df27a-53e7-469c-b9e5-be948a9e814a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:11.267933 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0df27a-53e7-469c-b9e5-be948a9e814a None admin] RBAC: Adding query filter params (role.id=a7d20885b6d94928a34b3cc5b3a00006, scope.project.id=23c9d26ac78646c79204b87bd5332e2c, user.id=0fad2691baf0472d99015f9f123f8d32) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:01:11.268254 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0df27a-53e7-469c-b9e5-be948a9e814a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:11.269733 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0df27a-53e7-469c-b9e5-be948a9e814a None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:11.280008 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 77/154] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Jul 25 12:01:11 2022] GET /identity/v3/role_assignments?user.id=0fad2691baf0472d99015f9f123f8d32&scope.project.id=23c9d26ac78646c79204b87bd5332e2c&role.id=a7d20885b6d94928a34b3cc5b3a00006 => generated 608 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 12:01:11.283719 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:11.328035 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36caf3fa-c9ec-425e-9fad-a794b1eda884 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:11.394769 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3fd0ae-39bd-4708-bd40-0aebf5fd8a04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:11.396120 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3fd0ae-39bd-4708-bd40-0aebf5fd8a04 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:11.396621 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3fd0ae-39bd-4708-bd40-0aebf5fd8a04 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:11.397116 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3fd0ae-39bd-4708-bd40-0aebf5fd8a04 None admin] PATH_INFO: `/v3/projects/23c9d26ac78646c79204b87bd5332e2c` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:11.398528 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3fd0ae-39bd-4708-bd40-0aebf5fd8a04 None admin] RBAC: Authorizing `identity:get_project(project_id=23c9d26ac78646c79204b87bd5332e2c)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:11.400231 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3fd0ae-39bd-4708-bd40-0aebf5fd8a04 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:11.402163 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 78/155] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Jul 25 12:01:11 2022] GET /identity/v3/projects/23c9d26ac78646c79204b87bd5332e2c => generated 298 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 12:01:11.408295 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0df27a-53e7-469c-b9e5-be948a9e814a None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:11.490443 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:11.550448 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0df27a-53e7-469c-b9e5-be948a9e814a None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:11.595070 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d32bce7-3db5-42ae-85fa-fb172f55e3b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:11.595960 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d32bce7-3db5-42ae-85fa-fb172f55e3b2 None admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:11.596251 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d32bce7-3db5-42ae-85fa-fb172f55e3b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:11.596537 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d32bce7-3db5-42ae-85fa-fb172f55e3b2 None admin] PATH_INFO: `/v3/projects/23c9d26ac78646c79204b87bd5332e2c/users/77064966b9d74fceaa268d13feebb7c1/roles/716f62e98b0c4df69e1441697434334c` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:11.598551 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d32bce7-3db5-42ae-85fa-fb172f55e3b2 None admin] RBAC: Authorizing `identity:create_grant(project_id=23c9d26ac78646c79204b87bd5332e2c, user_id=77064966b9d74fceaa268d13feebb7c1, role_id=716f62e98b0c4df69e1441697434334c)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:11.600034 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d32bce7-3db5-42ae-85fa-fb172f55e3b2 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:11.628109 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 78/156] 10.222.0.42 () {58 vars in 1439 bytes} [Mon Jul 25 12:01:11 2022] PUT /identity/v3/projects/23c9d26ac78646c79204b87bd5332e2c/users/77064966b9d74fceaa268d13feebb7c1/roles/716f62e98b0c4df69e1441697434334c => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:01:11.838792 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1b1174-7387-47b2-9756-ff1817d6fc3e None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:11.839450 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1b1174-7387-47b2-9756-ff1817d6fc3e None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:11.839960 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1b1174-7387-47b2-9756-ff1817d6fc3e None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:11.841547 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 79/157] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 25 12:01:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:01:11.847801 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a255b87-f3ea-4ac8-a6e6-71319c2ed265 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:11.848100 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a255b87-f3ea-4ac8-a6e6-71319c2ed265 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:11.848319 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a255b87-f3ea-4ac8-a6e6-71319c2ed265 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:11.877474 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-5a255b87-f3ea-4ac8-a6e6-71319c2ed265 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:01:11.878714 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-5a255b87-f3ea-4ac8-a6e6-71319c2ed265 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:01:11.969846 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 79/158] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 25 12:01:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 25 12:01:12.012719 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1b1174-7387-47b2-9756-ff1817d6fc3e None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:12.158827 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1b1174-7387-47b2-9756-ff1817d6fc3e None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:12.204908 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d4181a-6667-43b1-bd6b-8de49668fb5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:12.205721 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d4181a-6667-43b1-bd6b-8de49668fb5b None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:12.206012 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d4181a-6667-43b1-bd6b-8de49668fb5b None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:12.206293 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d4181a-6667-43b1-bd6b-8de49668fb5b None admin] PATH_INFO: `/v3/roles/member` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:12.220779 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d4181a-6667-43b1-bd6b-8de49668fb5b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 12:01:12.228618 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-b5d4181a-6667-43b1-bd6b-8de49668fb5b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 25 12:01:12.228618 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:01:12.228618 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:01:12.228618 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:01:12.228618 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:01:12.228618 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:01:12.228618 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:01:12.228618 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:01:12.228618 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:01:12.228618 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:01:12.228618 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:01:12.228618 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:01:12.228618 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 12:01:12.228618 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 12:01:12.228618 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 12:01:12.228618 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 12:01:12.229604 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 12:01:12.229604 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 12:01:12.229604 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 12:01:12.229604 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 12:01:12.229604 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:01:12.229604 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:01:12.229604 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:01:12.229604 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:01:12.229604 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:01:12.229604 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:01:12.229604 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:01:12.229604 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 25 12:01:12.229604 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:01:12.229604 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:01:12.229604 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:01:12.229604 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:01:12.230448 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:01:12.230448 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:01:12.230448 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:01:12.230448 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:01:12.230448 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 12:01:12.230448 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 12:01:12.230448 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 12:01:12.230448 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 12:01:12.230448 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 12:01:12.230448 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 12:01:12.230448 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 25 12:01:12.230448 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 25 12:01:12.230448 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 80/159] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Jul 25 12:01:12 2022] GET /identity/v3/roles/member => generated 84 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:01:12.240656 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a255b87-f3ea-4ac8-a6e6-71319c2ed265 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:12.359955 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:12.396254 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a255b87-f3ea-4ac8-a6e6-71319c2ed265 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:12.455201 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d47c5ae-4da1-4709-a720-2549f85e0c19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:12.455830 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d47c5ae-4da1-4709-a720-2549f85e0c19 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:12.456121 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d47c5ae-4da1-4709-a720-2549f85e0c19 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:12.456411 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d47c5ae-4da1-4709-a720-2549f85e0c19 None admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:12.457177 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d47c5ae-4da1-4709-a720-2549f85e0c19 None admin] RBAC: Adding query filter params (name=member) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:01:12.457496 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d47c5ae-4da1-4709-a720-2549f85e0c19 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:12.458889 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d47c5ae-4da1-4709-a720-2549f85e0c19 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:12.472648 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 80/160] 10.222.0.42 () {56 vars in 1101 bytes} [Mon Jul 25 12:01:12 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:01:12.480639 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d4181a-6667-43b1-bd6b-8de49668fb5b None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:12.491878 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:12.638040 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d4181a-6667-43b1-bd6b-8de49668fb5b None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:12.691134 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e6464b-f183-4780-a8a3-932b90c928be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:12.692057 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e6464b-f183-4780-a8a3-932b90c928be None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:12.692347 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e6464b-f183-4780-a8a3-932b90c928be None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:12.692634 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e6464b-f183-4780-a8a3-932b90c928be None admin] PATH_INFO: `/v3/users/b66651fed8eb43758d5d02472aa1ffaf` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:12.696292 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e6464b-f183-4780-a8a3-932b90c928be None admin] RBAC: Authorizing `identity:get_user(user_id=b66651fed8eb43758d5d02472aa1ffaf)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:12.697785 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e6464b-f183-4780-a8a3-932b90c928be None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:12.699448 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 81/161] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Jul 25 12:01:12 2022] GET /identity/v3/users/b66651fed8eb43758d5d02472aa1ffaf => generated 275 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:01:12.705662 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d47c5ae-4da1-4709-a720-2549f85e0c19 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:12.706112 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e454218-148b-4a39-ab8b-d96acc695c26 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:12.706409 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e454218-148b-4a39-ab8b-d96acc695c26 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:12.706694 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e454218-148b-4a39-ab8b-d96acc695c26 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:12.708233 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 82/162] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 25 12:01:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:01:12.713461 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8af0225-e9a9-4303-8353-4a148734b652 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:12.713760 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8af0225-e9a9-4303-8353-4a148734b652 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:12.714049 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8af0225-e9a9-4303-8353-4a148734b652 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:12.743869 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-f8af0225-e9a9-4303-8353-4a148734b652 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:01:12.745080 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-f8af0225-e9a9-4303-8353-4a148734b652 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:01:12.868928 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 83/163] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 25 12:01:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 25 12:01:12.885390 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d47c5ae-4da1-4709-a720-2549f85e0c19 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:12.901978 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8af0225-e9a9-4303-8353-4a148734b652 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:12.904773 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:12.917608 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:12.952834 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db17d02-4995-4d8c-84dc-318b614e9e56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:12.953512 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db17d02-4995-4d8c-84dc-318b614e9e56 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:12.953803 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db17d02-4995-4d8c-84dc-318b614e9e56 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:12.954082 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db17d02-4995-4d8c-84dc-318b614e9e56 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:13.040057 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db17d02-4995-4d8c-84dc-318b614e9e56 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:13.041467 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db17d02-4995-4d8c-84dc-318b614e9e56 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:13.082692 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8af0225-e9a9-4303-8353-4a148734b652 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:13.150140 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbedd0f7-83c0-4353-8ed7-9cedb18d0eda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:13.151214 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbedd0f7-83c0-4353-8ed7-9cedb18d0eda None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:13.151679 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbedd0f7-83c0-4353-8ed7-9cedb18d0eda None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:13.152130 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbedd0f7-83c0-4353-8ed7-9cedb18d0eda None admin] PATH_INFO: `/v3/roles/member` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:13.172046 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbedd0f7-83c0-4353-8ed7-9cedb18d0eda None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 12:01:13.180476 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-dbedd0f7-83c0-4353-8ed7-9cedb18d0eda None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 25 12:01:13.180476 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:01:13.180476 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:01:13.180476 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:01:13.180476 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:01:13.180476 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:01:13.180476 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:01:13.180476 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:01:13.180476 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:01:13.180476 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:01:13.180476 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:01:13.180476 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:01:13.180476 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 12:01:13.180476 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 12:01:13.180476 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 12:01:13.180476 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 12:01:13.181521 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 12:01:13.181521 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 12:01:13.181521 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 12:01:13.181521 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 12:01:13.181521 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:01:13.181521 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:01:13.181521 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:01:13.181521 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:01:13.181521 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:01:13.181521 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:01:13.181521 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:01:13.181521 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 25 12:01:13.181521 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:01:13.181521 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:01:13.181521 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:01:13.181521 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:01:13.182503 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:01:13.182503 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:01:13.182503 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:01:13.182503 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:01:13.182503 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 12:01:13.182503 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 12:01:13.182503 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 12:01:13.182503 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 12:01:13.182503 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 12:01:13.182503 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 12:01:13.182503 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 25 12:01:13.182503 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 25 12:01:13.188603 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 84/164] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Jul 25 12:01:12 2022] GET /identity/v3/roles/member => generated 84 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 12:01:13.190703 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaef7152-444e-40c6-9896-e0e447d5b2cc None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:13.190822 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaef7152-444e-40c6-9896-e0e447d5b2cc None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:13.195527 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaef7152-444e-40c6-9896-e0e447d5b2cc None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:13.196214 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 85/165] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 25 12:01:13 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:01:13.196858 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaef7152-444e-40c6-9896-e0e447d5b2cc None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:13.203104 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 81/166] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Jul 25 12:01:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 498 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 12:01:13.213830 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14349a9a-1c47-49f5-923e-3b215d15f036 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:13.214318 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14349a9a-1c47-49f5-923e-3b215d15f036 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:13.214773 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14349a9a-1c47-49f5-923e-3b215d15f036 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:13.235690 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-14349a9a-1c47-49f5-923e-3b215d15f036 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:01:13.237778 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-14349a9a-1c47-49f5-923e-3b215d15f036 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:01:13.249658 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:13.351093 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:13.388055 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 82/167] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 25 12:01:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 25 12:01:13.388991 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14349a9a-1c47-49f5-923e-3b215d15f036 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:13.448212 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaef7152-444e-40c6-9896-e0e447d5b2cc None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:13.561434 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdebe0f6-d30b-4acc-93a7-a28706316446 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:13.562228 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdebe0f6-d30b-4acc-93a7-a28706316446 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:13.562406 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdebe0f6-d30b-4acc-93a7-a28706316446 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:13.562711 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdebe0f6-d30b-4acc-93a7-a28706316446 None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:13.563461 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdebe0f6-d30b-4acc-93a7-a28706316446 None admin] RBAC: Adding query filter params (name=member) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:01:13.563796 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdebe0f6-d30b-4acc-93a7-a28706316446 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:13.565773 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdebe0f6-d30b-4acc-93a7-a28706316446 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:13.587316 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 86/168] 10.222.0.42 () {56 vars in 1101 bytes} [Mon Jul 25 12:01:13 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 391 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:01:13.588283 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdebe0f6-d30b-4acc-93a7-a28706316446 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:13.592688 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14349a9a-1c47-49f5-923e-3b215d15f036 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:13.650283 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0567929e-6c4c-4c87-a81c-139c4efdb410 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:13.651968 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0567929e-6c4c-4c87-a81c-139c4efdb410 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:13.652481 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0567929e-6c4c-4c87-a81c-139c4efdb410 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:13.653003 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0567929e-6c4c-4c87-a81c-139c4efdb410 None admin] PATH_INFO: `/v3/projects/23c9d26ac78646c79204b87bd5332e2c` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:13.654456 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0567929e-6c4c-4c87-a81c-139c4efdb410 None admin] RBAC: Authorizing `identity:get_project(project_id=23c9d26ac78646c79204b87bd5332e2c)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:13.656174 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0567929e-6c4c-4c87-a81c-139c4efdb410 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:13.658762 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 83/169] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Jul 25 12:01:13 2022] GET /identity/v3/projects/23c9d26ac78646c79204b87bd5332e2c => generated 298 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:01:13.663567 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0567929e-6c4c-4c87-a81c-139c4efdb410 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:13.689915 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:13.806080 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdebe0f6-d30b-4acc-93a7-a28706316446 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:13.832516 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0567929e-6c4c-4c87-a81c-139c4efdb410 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:13.851012 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:13.862125 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31afb348-4a24-4278-b864-1c0b5ad34b58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:13.862954 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31afb348-4a24-4278-b864-1c0b5ad34b58 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:13.863246 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31afb348-4a24-4278-b864-1c0b5ad34b58 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:13.863533 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31afb348-4a24-4278-b864-1c0b5ad34b58 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:13.891035 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539bb671-8acd-4541-b37c-ec2079b17843 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:13.891445 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31afb348-4a24-4278-b864-1c0b5ad34b58 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 12:01:13.892093 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539bb671-8acd-4541-b37c-ec2079b17843 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:13.892419 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539bb671-8acd-4541-b37c-ec2079b17843 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:13.892707 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539bb671-8acd-4541-b37c-ec2079b17843 None admin] PATH_INFO: `/v3/users/b66651fed8eb43758d5d02472aa1ffaf` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:13.893945 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539bb671-8acd-4541-b37c-ec2079b17843 None admin] RBAC: Authorizing `identity:get_user(user_id=b66651fed8eb43758d5d02472aa1ffaf)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:13.895400 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539bb671-8acd-4541-b37c-ec2079b17843 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:13.897382 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 84/170] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Jul 25 12:01:13 2022] GET /identity/v3/users/b66651fed8eb43758d5d02472aa1ffaf => generated 275 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:01:13.898160 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539bb671-8acd-4541-b37c-ec2079b17843 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:13.910276 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-31afb348-4a24-4278-b864-1c0b5ad34b58 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 12:01:13.910276 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:01:13.910276 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:01:13.910276 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:01:13.910276 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:01:13.910276 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:01:13.910276 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:01:13.910276 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:01:13.910276 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:01:13.910276 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:01:13.910276 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:01:13.910276 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:01:13.910276 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 12:01:13.910276 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 12:01:13.910276 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 12:01:13.910276 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 12:01:13.911184 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 12:01:13.911184 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 12:01:13.911184 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 12:01:13.911184 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 12:01:13.911184 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:01:13.911184 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:01:13.911184 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:01:13.911184 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:01:13.911184 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:01:13.911184 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:01:13.911184 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:01:13.911184 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 25 12:01:13.911184 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:01:13.911184 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:01:13.911184 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:01:13.911184 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:01:13.911983 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:01:13.911983 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:01:13.911983 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:01:13.911983 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:01:13.911983 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 12:01:13.911983 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 12:01:13.911983 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 12:01:13.911983 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 12:01:13.911983 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 12:01:13.911983 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 12:01:13.911983 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 12:01:13.911983 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 25 12:01:13.911983 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 87/171] 10.222.0.42 () {56 vars in 1114 bytes} [Mon Jul 25 12:01:13 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 324 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:01:13.913225 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31afb348-4a24-4278-b864-1c0b5ad34b58 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:13.993561 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:14.014844 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539bb671-8acd-4541-b37c-ec2079b17843 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:14.027832 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31afb348-4a24-4278-b864-1c0b5ad34b58 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:14.057582 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cd72f7-847e-444a-9367-aa2309e27a17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:14.058248 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cd72f7-847e-444a-9367-aa2309e27a17 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:14.058540 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cd72f7-847e-444a-9367-aa2309e27a17 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:14.058826 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cd72f7-847e-444a-9367-aa2309e27a17 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:14.059759 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03cd72f7-847e-444a-9367-aa2309e27a17 None admin] RBAC: Adding query filter params (role.id=472313ae69174578824df0bed6390744, scope.project.id=23c9d26ac78646c79204b87bd5332e2c, user.id=b66651fed8eb43758d5d02472aa1ffaf) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:01:14.060079 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03cd72f7-847e-444a-9367-aa2309e27a17 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:14.061578 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03cd72f7-847e-444a-9367-aa2309e27a17 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:14.067877 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 85/172] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Jul 25 12:01:13 2022] GET /identity/v3/role_assignments?user.id=b66651fed8eb43758d5d02472aa1ffaf&scope.project.id=23c9d26ac78646c79204b87bd5332e2c&role.id=472313ae69174578824df0bed6390744 => generated 608 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:01:14.069259 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4c2a65-27f1-48c7-ba8a-df5073b56b6d None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:14.069559 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4c2a65-27f1-48c7-ba8a-df5073b56b6d None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:14.069848 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4c2a65-27f1-48c7-ba8a-df5073b56b6d None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:14.071095 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 86/173] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 25 12:01:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:01:14.071989 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4c2a65-27f1-48c7-ba8a-df5073b56b6d None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:14.075741 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e417bd8e-6056-4375-a560-d6e4fa58db5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:14.077345 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e417bd8e-6056-4375-a560-d6e4fa58db5d None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:14.077642 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e417bd8e-6056-4375-a560-d6e4fa58db5d None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:14.077931 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e417bd8e-6056-4375-a560-d6e4fa58db5d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:14.199204 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:14.225923 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e417bd8e-6056-4375-a560-d6e4fa58db5d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:14.227425 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e417bd8e-6056-4375-a560-d6e4fa58db5d None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:14.268117 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4c2a65-27f1-48c7-ba8a-df5073b56b6d None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:14.320384 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed9cb09-a2e9-4294-85cd-4f47dd83cb3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:14.321333 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed9cb09-a2e9-4294-85cd-4f47dd83cb3f None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:14.321771 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed9cb09-a2e9-4294-85cd-4f47dd83cb3f None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:14.322210 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed9cb09-a2e9-4294-85cd-4f47dd83cb3f None admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:14.322977 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed9cb09-a2e9-4294-85cd-4f47dd83cb3f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:01:14.323299 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed9cb09-a2e9-4294-85cd-4f47dd83cb3f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:14.324719 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed9cb09-a2e9-4294-85cd-4f47dd83cb3f None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:14.335563 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 87/174] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Jul 25 12:01:14 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:01:14.339634 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd01e3e-f8c5-4c75-943a-66a624db81ee None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:14.339634 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd01e3e-f8c5-4c75-943a-66a624db81ee None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:14.339634 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd01e3e-f8c5-4c75-943a-66a624db81ee None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:14.339634 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 88/175] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 25 12:01:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:01:14.340027 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa721ee-827b-4fc6-a5ac-1c4c2f281c9a None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:14.340095 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa721ee-827b-4fc6-a5ac-1c4c2f281c9a None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:14.340344 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa721ee-827b-4fc6-a5ac-1c4c2f281c9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:14.353999 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:14.360620 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-2aa721ee-827b-4fc6-a5ac-1c4c2f281c9a None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:01:14.363765 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-2aa721ee-827b-4fc6-a5ac-1c4c2f281c9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:01:14.374064 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 88/176] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Jul 25 12:01:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 462 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 12:01:14.376744 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa89448c-f053-42c2-91cd-649039096480 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:14.377253 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa89448c-f053-42c2-91cd-649039096480 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:14.377759 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa89448c-f053-42c2-91cd-649039096480 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:14.379267 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 89/177] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 25 12:01:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:01:14.380554 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef2defa-73a6-47da-9cd9-66960149a530 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:14.384845 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef2defa-73a6-47da-9cd9-66960149a530 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:14.385393 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef2defa-73a6-47da-9cd9-66960149a530 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:14.405127 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-8ef2defa-73a6-47da-9cd9-66960149a530 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:01:14.406885 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-8ef2defa-73a6-47da-9cd9-66960149a530 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:01:14.440107 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 89/178] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 25 12:01:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 12:01:14.441638 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa721ee-827b-4fc6-a5ac-1c4c2f281c9a None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:14.498827 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 90/179] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 25 12:01:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 12:01:14.498827 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef2defa-73a6-47da-9cd9-66960149a530 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:14.518474 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:14.558407 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa721ee-827b-4fc6-a5ac-1c4c2f281c9a None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:14.592572 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2744c596-842d-4ab7-9268-5bc20d029da2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:14.593444 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2744c596-842d-4ab7-9268-5bc20d029da2 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:14.593731 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2744c596-842d-4ab7-9268-5bc20d029da2 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:14.594010 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2744c596-842d-4ab7-9268-5bc20d029da2 None admin] PATH_INFO: `/v3/users/b66651fed8eb43758d5d02472aa1ffaf` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:14.595266 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2744c596-842d-4ab7-9268-5bc20d029da2 None admin] RBAC: Authorizing `identity:get_user(user_id=b66651fed8eb43758d5d02472aa1ffaf)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:14.597254 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2744c596-842d-4ab7-9268-5bc20d029da2 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:14.598873 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 90/180] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Jul 25 12:01:14 2022] GET /identity/v3/users/b66651fed8eb43758d5d02472aa1ffaf => generated 275 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:01:14.600175 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693c3d30-8962-48bc-9083-944666388aaa None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:14.600471 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693c3d30-8962-48bc-9083-944666388aaa None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:14.600755 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693c3d30-8962-48bc-9083-944666388aaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:14.606379 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef2defa-73a6-47da-9cd9-66960149a530 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:14.607839 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:14.618690 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-693c3d30-8962-48bc-9083-944666388aaa None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:01:14.619921 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-693c3d30-8962-48bc-9083-944666388aaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:01:14.640370 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b6a306-fbcc-4f76-bb57-d6d41a30556f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:14.641226 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b6a306-fbcc-4f76-bb57-d6d41a30556f None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:14.641507 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b6a306-fbcc-4f76-bb57-d6d41a30556f None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:14.641783 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b6a306-fbcc-4f76-bb57-d6d41a30556f None admin] PATH_INFO: `/v3/projects/946e20d010884b9f84dea264e0a522ea` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:14.643446 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b6a306-fbcc-4f76-bb57-d6d41a30556f None admin] RBAC: Authorizing `identity:get_project(project_id=946e20d010884b9f84dea264e0a522ea)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:14.644914 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b6a306-fbcc-4f76-bb57-d6d41a30556f None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:14.646609 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 91/181] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Jul 25 12:01:14 2022] GET /identity/v3/projects/946e20d010884b9f84dea264e0a522ea => generated 312 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:01:14.648417 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b6a306-fbcc-4f76-bb57-d6d41a30556f None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:14.685434 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 91/182] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 25 12:01:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 12:01:14.686326 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693c3d30-8962-48bc-9083-944666388aaa None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:14.737991 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:14.754669 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b6a306-fbcc-4f76-bb57-d6d41a30556f None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:14.786975 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693c3d30-8962-48bc-9083-944666388aaa None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:14.788174 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7250e9c9-3ecc-4e50-be99-bee518d740f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:14.788975 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7250e9c9-3ecc-4e50-be99-bee518d740f0 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:14.789259 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7250e9c9-3ecc-4e50-be99-bee518d740f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:14.789534 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7250e9c9-3ecc-4e50-be99-bee518d740f0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:14.799970 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7250e9c9-3ecc-4e50-be99-bee518d740f0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 12:01:14.807653 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-7250e9c9-3ecc-4e50-be99-bee518d740f0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 12:01:14.807653 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:01:14.807653 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:01:14.807653 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:01:14.807653 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:01:14.807653 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:01:14.807653 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:01:14.807653 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:01:14.807653 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:01:14.807653 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:01:14.807653 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:01:14.807653 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:01:14.807653 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 12:01:14.807653 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 12:01:14.807653 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 12:01:14.807653 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 12:01:14.808564 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 12:01:14.808564 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 12:01:14.808564 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 12:01:14.808564 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 12:01:14.808564 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:01:14.808564 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:01:14.808564 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:01:14.808564 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:01:14.808564 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:01:14.808564 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:01:14.808564 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:01:14.808564 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 25 12:01:14.808564 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:01:14.808564 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:01:14.808564 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:01:14.808564 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:01:14.809377 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:01:14.809377 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:01:14.809377 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:01:14.809377 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:01:14.809377 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 12:01:14.809377 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 12:01:14.809377 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 12:01:14.809377 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 12:01:14.809377 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 12:01:14.809377 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 12:01:14.809377 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 12:01:14.809377 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 25 12:01:14.809377 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 92/183] 10.222.0.42 () {56 vars in 1096 bytes} [Mon Jul 25 12:01:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:01:14.810037 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4eb9eba-63b9-4172-b22b-efc5ac43359f None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:14.810320 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4eb9eba-63b9-4172-b22b-efc5ac43359f None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:14.810650 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4eb9eba-63b9-4172-b22b-efc5ac43359f None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:14.811856 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 93/184] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 25 12:01:14 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:01:14.813722 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4eb9eba-63b9-4172-b22b-efc5ac43359f None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:14.826303 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885cbf46-2828-4aa7-aca5-5b2d56c5744f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:14.827102 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885cbf46-2828-4aa7-aca5-5b2d56c5744f None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:14.827392 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885cbf46-2828-4aa7-aca5-5b2d56c5744f None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:14.827705 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885cbf46-2828-4aa7-aca5-5b2d56c5744f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:14.828875 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885cbf46-2828-4aa7-aca5-5b2d56c5744f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:14.830945 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885cbf46-2828-4aa7-aca5-5b2d56c5744f None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:14.832694 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 92/185] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jul 25 12:01:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:01:14.833447 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885cbf46-2828-4aa7-aca5-5b2d56c5744f None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:14.835944 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:14.919369 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4eb9eba-63b9-4172-b22b-efc5ac43359f None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:14.931262 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885cbf46-2828-4aa7-aca5-5b2d56c5744f None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:14.951741 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5c94ae-d062-4ab1-98fe-296705a4c762 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:14.952396 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5c94ae-d062-4ab1-98fe-296705a4c762 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:14.952685 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5c94ae-d062-4ab1-98fe-296705a4c762 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:14.952966 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5c94ae-d062-4ab1-98fe-296705a4c762 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:14.965847 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bcadaa-afb0-4ca4-b3d1-1e5ee6e610b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:14.966506 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bcadaa-afb0-4ca4-b3d1-1e5ee6e610b9 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:14.966796 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bcadaa-afb0-4ca4-b3d1-1e5ee6e610b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:14.967080 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bcadaa-afb0-4ca4-b3d1-1e5ee6e610b9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:14.967497 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:14.967968 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3bcadaa-afb0-4ca4-b3d1-1e5ee6e610b9 None admin] RBAC: Adding query filter params (role.id=472313ae69174578824df0bed6390744, scope.project.id=946e20d010884b9f84dea264e0a522ea, user.id=b66651fed8eb43758d5d02472aa1ffaf) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:01:14.968288 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3bcadaa-afb0-4ca4-b3d1-1e5ee6e610b9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:14.969885 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3bcadaa-afb0-4ca4-b3d1-1e5ee6e610b9 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:14.976393 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 93/186] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Jul 25 12:01:14 2022] GET /identity/v3/role_assignments?user.id=b66651fed8eb43758d5d02472aa1ffaf&scope.project.id=946e20d010884b9f84dea264e0a522ea&role.id=472313ae69174578824df0bed6390744 => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:01:14.977174 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bcadaa-afb0-4ca4-b3d1-1e5ee6e610b9 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:15.041142 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5c94ae-d062-4ab1-98fe-296705a4c762 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:15.042650 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5c94ae-d062-4ab1-98fe-296705a4c762 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:15.051474 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:15.094826 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bcadaa-afb0-4ca4-b3d1-1e5ee6e610b9 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:15.139165 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043b0d08-ed0d-4e7d-933a-c2be4cfd6667 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:15.140025 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043b0d08-ed0d-4e7d-933a-c2be4cfd6667 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:15.140318 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043b0d08-ed0d-4e7d-933a-c2be4cfd6667 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:15.140601 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043b0d08-ed0d-4e7d-933a-c2be4cfd6667 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:15.152750 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043b0d08-ed0d-4e7d-933a-c2be4cfd6667 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 12:01:15.158735 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 94/187] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Jul 25 12:01:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 12:01:15.160579 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5c94ae-d062-4ab1-98fe-296705a4c762 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:15.161530 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-043b0d08-ed0d-4e7d-933a-c2be4cfd6667 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 12:01:15.161530 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:01:15.161530 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:01:15.161530 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:01:15.161530 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:01:15.161530 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:01:15.161530 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:01:15.161530 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:01:15.161530 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:01:15.161530 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:01:15.161530 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:01:15.161530 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:01:15.161530 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 12:01:15.161530 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 12:01:15.161530 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 12:01:15.161530 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 12:01:15.162462 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 12:01:15.162462 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 12:01:15.162462 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 12:01:15.162462 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 12:01:15.162462 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:01:15.162462 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:01:15.162462 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:01:15.162462 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:01:15.162462 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:01:15.162462 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:01:15.162462 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:01:15.162462 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jul 25 12:01:15.162462 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:01:15.162462 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:01:15.162462 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:01:15.162462 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:01:15.163236 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:01:15.163236 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:01:15.163236 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:01:15.163236 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:01:15.163236 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 12:01:15.163236 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 12:01:15.163236 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 12:01:15.163236 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 12:01:15.163236 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 12:01:15.163236 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 12:01:15.163236 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 12:01:15.163236 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jul 25 12:01:15.163236 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 94/188] 10.222.0.42 () {56 vars in 1114 bytes} [Mon Jul 25 12:01:14 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:01:15.164618 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1fa7cf-64eb-4b68-abde-ff74e7d9ed8a None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:15.165163 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1fa7cf-64eb-4b68-abde-ff74e7d9ed8a None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:15.165445 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1fa7cf-64eb-4b68-abde-ff74e7d9ed8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:15.185378 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-6d1fa7cf-64eb-4b68-abde-ff74e7d9ed8a None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:01:15.186568 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-6d1fa7cf-64eb-4b68-abde-ff74e7d9ed8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:01:15.217743 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:15.258614 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 95/189] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 25 12:01:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 12:01:15.259449 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1fa7cf-64eb-4b68-abde-ff74e7d9ed8a None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:15.281861 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5c94ae-d062-4ab1-98fe-296705a4c762 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:15.314813 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:15.316112 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498246ba-c0fc-47b7-b7ab-8a3eae7978ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:15.316951 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498246ba-c0fc-47b7-b7ab-8a3eae7978ef None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:15.317239 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498246ba-c0fc-47b7-b7ab-8a3eae7978ef None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:15.317533 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498246ba-c0fc-47b7-b7ab-8a3eae7978ef None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:15.318292 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498246ba-c0fc-47b7-b7ab-8a3eae7978ef None admin] RBAC: Adding query filter params (name=admin) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:01:15.318615 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498246ba-c0fc-47b7-b7ab-8a3eae7978ef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:15.320079 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498246ba-c0fc-47b7-b7ab-8a3eae7978ef None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:15.331907 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 95/190] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Jul 25 12:01:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:01:15.332601 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498246ba-c0fc-47b7-b7ab-8a3eae7978ef None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:15.361510 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1fa7cf-64eb-4b68-abde-ff74e7d9ed8a None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:15.395715 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f54f833-3845-4baa-9251-82aeec0680a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:15.396381 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f54f833-3845-4baa-9251-82aeec0680a9 None admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:15.396665 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f54f833-3845-4baa-9251-82aeec0680a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:15.397017 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f54f833-3845-4baa-9251-82aeec0680a9 None admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:15.397729 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f54f833-3845-4baa-9251-82aeec0680a9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:15.399171 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f54f833-3845-4baa-9251-82aeec0680a9 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:15.420575 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 96/191] 10.222.0.42 () {60 vars in 1132 bytes} [Mon Jul 25 12:01:15 2022] POST /identity/v3/users => generated 297 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:01:15.421391 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f54f833-3845-4baa-9251-82aeec0680a9 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:15.444580 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498246ba-c0fc-47b7-b7ab-8a3eae7978ef None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:15.452795 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:15.485398 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f151570b-448e-450b-aa47-6caefe2a5ad4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:15.486655 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f151570b-448e-450b-aa47-6caefe2a5ad4 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:15.486949 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f151570b-448e-450b-aa47-6caefe2a5ad4 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:15.487242 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f151570b-448e-450b-aa47-6caefe2a5ad4 None admin] PATH_INFO: `/v3/projects/23c9d26ac78646c79204b87bd5332e2c` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:15.488553 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f151570b-448e-450b-aa47-6caefe2a5ad4 None admin] RBAC: Authorizing `identity:get_project(project_id=23c9d26ac78646c79204b87bd5332e2c)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:15.490048 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f151570b-448e-450b-aa47-6caefe2a5ad4 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:15.492217 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 96/192] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Jul 25 12:01:15 2022] GET /identity/v3/projects/23c9d26ac78646c79204b87bd5332e2c => generated 298 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:01:15.492690 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f151570b-448e-450b-aa47-6caefe2a5ad4 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:15.574791 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f54f833-3845-4baa-9251-82aeec0680a9 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:15.588605 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:15.616052 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f151570b-448e-450b-aa47-6caefe2a5ad4 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:15.617736 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a456b5f2-288d-4563-a744-5d5d30720c40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:15.618430 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a456b5f2-288d-4563-a744-5d5d30720c40 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:15.620205 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a456b5f2-288d-4563-a744-5d5d30720c40 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:15.620205 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a456b5f2-288d-4563-a744-5d5d30720c40 None admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:15.620205 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a456b5f2-288d-4563-a744-5d5d30720c40 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:01:15.620205 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a456b5f2-288d-4563-a744-5d5d30720c40 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:15.621506 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a456b5f2-288d-4563-a744-5d5d30720c40 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:15.630766 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 97/193] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Jul 25 12:01:15 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:01:15.631472 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a456b5f2-288d-4563-a744-5d5d30720c40 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:15.663165 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc59db5-db64-4357-92e2-145b1a5f9f98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:15.664029 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc59db5-db64-4357-92e2-145b1a5f9f98 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:15.664479 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc59db5-db64-4357-92e2-145b1a5f9f98 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:15.664717 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc59db5-db64-4357-92e2-145b1a5f9f98 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:15.684992 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc59db5-db64-4357-92e2-145b1a5f9f98 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 12:01:15.693517 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-cdc59db5-db64-4357-92e2-145b1a5f9f98 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 25 12:01:15.693517 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:01:15.693517 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:01:15.693517 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:01:15.693517 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:01:15.693517 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:01:15.693517 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:01:15.693517 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:01:15.693517 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:01:15.693517 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:01:15.693517 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:01:15.693517 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:01:15.693517 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 12:01:15.693517 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 12:01:15.693517 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 12:01:15.693517 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 12:01:15.694418 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 12:01:15.694418 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 12:01:15.694418 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 12:01:15.694418 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 12:01:15.694418 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:01:15.694418 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:01:15.694418 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:01:15.694418 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:01:15.694418 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:01:15.694418 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:01:15.694418 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:01:15.694418 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 25 12:01:15.694418 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:01:15.694418 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:01:15.694418 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:01:15.694418 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:01:15.695370 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:01:15.695370 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:01:15.695370 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:01:15.695370 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:01:15.695370 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 12:01:15.695370 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 12:01:15.695370 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 12:01:15.695370 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 12:01:15.695370 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 12:01:15.695370 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 12:01:15.695370 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 25 12:01:15.695370 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 25 12:01:15.695370 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 97/194] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Jul 25 12:01:15 2022] GET /identity/v3/roles/reader => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:01:15.696120 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc59db5-db64-4357-92e2-145b1a5f9f98 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:15.741474 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:15.764078 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a456b5f2-288d-4563-a744-5d5d30720c40 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:15.814626 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46bbe95-624b-4fba-b6cd-30549d7c1a85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:15.815713 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46bbe95-624b-4fba-b6cd-30549d7c1a85 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:15.816304 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46bbe95-624b-4fba-b6cd-30549d7c1a85 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:15.816665 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46bbe95-624b-4fba-b6cd-30549d7c1a85 None admin] PATH_INFO: `/v3/users/6f9828f755914644aa268493e126bb00` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:15.823861 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc59db5-db64-4357-92e2-145b1a5f9f98 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:15.835237 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46bbe95-624b-4fba-b6cd-30549d7c1a85 None admin] RBAC: Authorizing `identity:get_user(user_id=6f9828f755914644aa268493e126bb00)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:15.837434 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46bbe95-624b-4fba-b6cd-30549d7c1a85 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:15.843569 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 98/195] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Jul 25 12:01:15 2022] GET /identity/v3/users/6f9828f755914644aa268493e126bb00 => generated 283 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:01:15.843569 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46bbe95-624b-4fba-b6cd-30549d7c1a85 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:15.866822 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b513b8-5ed6-4e3b-b5b4-68169ba6ad69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:15.867859 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b513b8-5ed6-4e3b-b5b4-68169ba6ad69 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:15.868371 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b513b8-5ed6-4e3b-b5b4-68169ba6ad69 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:15.868899 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b513b8-5ed6-4e3b-b5b4-68169ba6ad69 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:15.869935 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b513b8-5ed6-4e3b-b5b4-68169ba6ad69 None admin] RBAC: Adding query filter params (role.id=dea3ef72822043f3baf3bcc1be7c318f, scope.project.id=23c9d26ac78646c79204b87bd5332e2c, user.id=b66651fed8eb43758d5d02472aa1ffaf) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:01:15.870471 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b513b8-5ed6-4e3b-b5b4-68169ba6ad69 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:15.872237 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b513b8-5ed6-4e3b-b5b4-68169ba6ad69 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:15.878803 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:15.880317 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 98/196] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Jul 25 12:01:15 2022] GET /identity/v3/role_assignments?user.id=b66651fed8eb43758d5d02472aa1ffaf&scope.project.id=23c9d26ac78646c79204b87bd5332e2c&role.id=dea3ef72822043f3baf3bcc1be7c318f => generated 608 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:01:15.881571 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b513b8-5ed6-4e3b-b5b4-68169ba6ad69 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:15.957980 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46bbe95-624b-4fba-b6cd-30549d7c1a85 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:15.996186 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:16.005993 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2014e6f-bce4-4d8e-92c0-e5dd590178d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:16.009074 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2014e6f-bce4-4d8e-92c0-e5dd590178d9 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:16.009645 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2014e6f-bce4-4d8e-92c0-e5dd590178d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:16.010133 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b513b8-5ed6-4e3b-b5b4-68169ba6ad69 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:16.010622 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2014e6f-bce4-4d8e-92c0-e5dd590178d9 None admin] PATH_INFO: `/v3/users/6f9828f755914644aa268493e126bb00` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:16.013963 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2014e6f-bce4-4d8e-92c0-e5dd590178d9 None admin] RBAC: Authorizing `identity:get_user(user_id=6f9828f755914644aa268493e126bb00)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:16.017607 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2014e6f-bce4-4d8e-92c0-e5dd590178d9 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:16.026693 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 99/197] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Jul 25 12:01:15 2022] GET /identity/v3/users/6f9828f755914644aa268493e126bb00 => generated 283 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:01:16.028668 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2014e6f-bce4-4d8e-92c0-e5dd590178d9 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:16.076779 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d9d3a2-ee07-4ba8-8017-b3473b917bc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:16.077948 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d9d3a2-ee07-4ba8-8017-b3473b917bc6 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:16.078750 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d9d3a2-ee07-4ba8-8017-b3473b917bc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:16.079240 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d9d3a2-ee07-4ba8-8017-b3473b917bc6 None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:16.080394 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d9d3a2-ee07-4ba8-8017-b3473b917bc6 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:01:16.080993 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d9d3a2-ee07-4ba8-8017-b3473b917bc6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:16.082612 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d9d3a2-ee07-4ba8-8017-b3473b917bc6 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:16.094837 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 99/198] 10.222.0.42 () {56 vars in 1101 bytes} [Mon Jul 25 12:01:15 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:01:16.096162 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d9d3a2-ee07-4ba8-8017-b3473b917bc6 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:16.193930 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:16.196913 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2014e6f-bce4-4d8e-92c0-e5dd590178d9 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:16.229929 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d9d3a2-ee07-4ba8-8017-b3473b917bc6 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:16.236210 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c545198-131f-4071-8b4c-173b571faa87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:16.236869 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c545198-131f-4071-8b4c-173b571faa87 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:16.237152 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c545198-131f-4071-8b4c-173b571faa87 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:16.237430 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c545198-131f-4071-8b4c-173b571faa87 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:16.269543 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c954ec-30e8-47d0-a46d-31d020bd6b3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:16.270267 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c954ec-30e8-47d0-a46d-31d020bd6b3d None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:16.270564 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c954ec-30e8-47d0-a46d-31d020bd6b3d None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:16.270848 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c954ec-30e8-47d0-a46d-31d020bd6b3d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:16.304029 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:16.314662 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c545198-131f-4071-8b4c-173b571faa87 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:16.316181 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c545198-131f-4071-8b4c-173b571faa87 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:16.352154 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c954ec-30e8-47d0-a46d-31d020bd6b3d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:16.354083 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c954ec-30e8-47d0-a46d-31d020bd6b3d None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:16.442332 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:16.449429 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 100/199] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Jul 25 12:01:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 12:01:16.450836 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c545198-131f-4071-8b4c-173b571faa87 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:16.475414 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 100/200] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Jul 25 12:01:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 12:01:16.480509 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c954ec-30e8-47d0-a46d-31d020bd6b3d None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:16.544866 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:16.590113 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c954ec-30e8-47d0-a46d-31d020bd6b3d None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:16.619024 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c545198-131f-4071-8b4c-173b571faa87 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:16.627691 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51bb09e-2514-4ea5-8e1d-9f9befadce02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:16.629031 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51bb09e-2514-4ea5-8e1d-9f9befadce02 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:16.629499 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51bb09e-2514-4ea5-8e1d-9f9befadce02 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:16.630004 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51bb09e-2514-4ea5-8e1d-9f9befadce02 None admin] PATH_INFO: `/v3/projects/c980d1440d8f49049c8140fffd0d1e15` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:16.631402 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51bb09e-2514-4ea5-8e1d-9f9befadce02 None admin] RBAC: Authorizing `identity:get_project(project_id=c980d1440d8f49049c8140fffd0d1e15)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:16.635362 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51bb09e-2514-4ea5-8e1d-9f9befadce02 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:16.639843 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 101/201] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Jul 25 12:01:16 2022] GET /identity/v3/projects/c980d1440d8f49049c8140fffd0d1e15 => generated 302 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:01:16.640783 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51bb09e-2514-4ea5-8e1d-9f9befadce02 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:16.679042 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad660a8a-0025-4273-b8d8-634f075eae75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:16.680487 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad660a8a-0025-4273-b8d8-634f075eae75 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:16.683117 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad660a8a-0025-4273-b8d8-634f075eae75 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:16.683430 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad660a8a-0025-4273-b8d8-634f075eae75 None admin] PATH_INFO: `/v3/users/77064966b9d74fceaa268d13feebb7c1` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:16.684709 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad660a8a-0025-4273-b8d8-634f075eae75 None admin] RBAC: Authorizing `identity:get_user(user_id=77064966b9d74fceaa268d13feebb7c1)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:16.686164 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad660a8a-0025-4273-b8d8-634f075eae75 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:16.693819 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 101/202] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Jul 25 12:01:16 2022] GET /identity/v3/users/77064966b9d74fceaa268d13feebb7c1 => generated 289 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:01:16.694461 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad660a8a-0025-4273-b8d8-634f075eae75 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:16.757327 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51bb09e-2514-4ea5-8e1d-9f9befadce02 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:16.792730 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:16.819894 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:16.830665 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad660a8a-0025-4273-b8d8-634f075eae75 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:16.833258 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b4896c-2ab7-45a5-9a03-fdfea88ea2bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:16.835833 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b4896c-2ab7-45a5-9a03-fdfea88ea2bc None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:16.836126 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b4896c-2ab7-45a5-9a03-fdfea88ea2bc None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:16.836413 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b4896c-2ab7-45a5-9a03-fdfea88ea2bc None admin] PATH_INFO: `/v3/projects/c980d1440d8f49049c8140fffd0d1e15` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:16.839799 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b4896c-2ab7-45a5-9a03-fdfea88ea2bc None admin] RBAC: Authorizing `identity:get_project(project_id=c980d1440d8f49049c8140fffd0d1e15)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:16.841279 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b4896c-2ab7-45a5-9a03-fdfea88ea2bc None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:16.854070 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 102/203] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Jul 25 12:01:16 2022] GET /identity/v3/projects/c980d1440d8f49049c8140fffd0d1e15 => generated 302 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:01:16.854962 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b4896c-2ab7-45a5-9a03-fdfea88ea2bc None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:16.870599 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9ad8e5-5ff4-46f9-82d5-84918dd6138c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:16.871444 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9ad8e5-5ff4-46f9-82d5-84918dd6138c None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:16.871894 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9ad8e5-5ff4-46f9-82d5-84918dd6138c None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:16.872406 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9ad8e5-5ff4-46f9-82d5-84918dd6138c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:16.874038 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9ad8e5-5ff4-46f9-82d5-84918dd6138c None admin] RBAC: Adding query filter params (role.id=a7d20885b6d94928a34b3cc5b3a00006, scope.project.id=c980d1440d8f49049c8140fffd0d1e15, user.id=6f9828f755914644aa268493e126bb00) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:01:16.874750 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9ad8e5-5ff4-46f9-82d5-84918dd6138c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:16.876908 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9ad8e5-5ff4-46f9-82d5-84918dd6138c None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:16.883050 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 102/204] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Jul 25 12:01:16 2022] GET /identity/v3/role_assignments?user.id=6f9828f755914644aa268493e126bb00&scope.project.id=c980d1440d8f49049c8140fffd0d1e15&role.id=a7d20885b6d94928a34b3cc5b3a00006 => generated 259 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:01:16.884043 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9ad8e5-5ff4-46f9-82d5-84918dd6138c None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:17.001281 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9ad8e5-5ff4-46f9-82d5-84918dd6138c None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:17.036776 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:17.043354 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb348970-a245-4a6b-b707-1f1dd51c4313 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:17.044377 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb348970-a245-4a6b-b707-1f1dd51c4313 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:17.044907 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb348970-a245-4a6b-b707-1f1dd51c4313 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:17.045389 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb348970-a245-4a6b-b707-1f1dd51c4313 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:17.046545 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb348970-a245-4a6b-b707-1f1dd51c4313 None admin] RBAC: Adding query filter params (role.id=dea3ef72822043f3baf3bcc1be7c318f, scope.project.id=c980d1440d8f49049c8140fffd0d1e15, user.id=6f9828f755914644aa268493e126bb00) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:01:17.047118 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb348970-a245-4a6b-b707-1f1dd51c4313 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:17.049972 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb348970-a245-4a6b-b707-1f1dd51c4313 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:17.056487 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 103/205] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Jul 25 12:01:16 2022] GET /identity/v3/role_assignments?user.id=6f9828f755914644aa268493e126bb00&scope.project.id=c980d1440d8f49049c8140fffd0d1e15&role.id=dea3ef72822043f3baf3bcc1be7c318f => generated 259 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:01:17.119207 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b4896c-2ab7-45a5-9a03-fdfea88ea2bc None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:17.181311 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed7553b-f496-49e7-b369-2b449d6c2b67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:17.181915 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed7553b-f496-49e7-b369-2b449d6c2b67 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:17.182205 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed7553b-f496-49e7-b369-2b449d6c2b67 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:17.182487 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed7553b-f496-49e7-b369-2b449d6c2b67 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:17.289396 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:17.297587 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed7553b-f496-49e7-b369-2b449d6c2b67 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:17.299729 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed7553b-f496-49e7-b369-2b449d6c2b67 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:17.406530 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 103/206] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Jul 25 12:01:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 552 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 12:01:17.412036 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb348970-a245-4a6b-b707-1f1dd51c4313 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:17.534003 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb348970-a245-4a6b-b707-1f1dd51c4313 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:17.582209 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3292bc3a-d663-4ccb-9f92-030628205419 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:17.583139 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3292bc3a-d663-4ccb-9f92-030628205419 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:17.583432 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3292bc3a-d663-4ccb-9f92-030628205419 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:17.583741 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3292bc3a-d663-4ccb-9f92-030628205419 None admin] PATH_INFO: `/v3/projects/23c9d26ac78646c79204b87bd5332e2c` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:17.584980 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3292bc3a-d663-4ccb-9f92-030628205419 None admin] RBAC: Authorizing `identity:get_project(project_id=23c9d26ac78646c79204b87bd5332e2c)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:17.586437 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3292bc3a-d663-4ccb-9f92-030628205419 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:17.588253 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 104/207] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Jul 25 12:01:17 2022] GET /identity/v3/projects/23c9d26ac78646c79204b87bd5332e2c => generated 298 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 12:01:17.594144 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed7553b-f496-49e7-b369-2b449d6c2b67 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:17.697855 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed7553b-f496-49e7-b369-2b449d6c2b67 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:17.699613 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:17.750337 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf46e5a7-e6cd-47c7-aa9c-5c4700dc43e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:17.751011 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf46e5a7-e6cd-47c7-aa9c-5c4700dc43e4 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:17.751299 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf46e5a7-e6cd-47c7-aa9c-5c4700dc43e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:17.752132 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf46e5a7-e6cd-47c7-aa9c-5c4700dc43e4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:17.752987 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf46e5a7-e6cd-47c7-aa9c-5c4700dc43e4 None admin] RBAC: Adding query filter params (role.id=716f62e98b0c4df69e1441697434334c, scope.project.id=23c9d26ac78646c79204b87bd5332e2c, user.id=77064966b9d74fceaa268d13feebb7c1) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:01:17.753306 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf46e5a7-e6cd-47c7-aa9c-5c4700dc43e4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:17.754771 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf46e5a7-e6cd-47c7-aa9c-5c4700dc43e4 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:17.761302 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 104/208] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Jul 25 12:01:17 2022] GET /identity/v3/role_assignments?user.id=77064966b9d74fceaa268d13feebb7c1&scope.project.id=23c9d26ac78646c79204b87bd5332e2c&role.id=716f62e98b0c4df69e1441697434334c => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:01:17.862781 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c253b676-c2b2-4518-9765-2810515f1830 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:17.863328 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c253b676-c2b2-4518-9765-2810515f1830 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:17.863328 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c253b676-c2b2-4518-9765-2810515f1830 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:17.865042 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 105/209] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 25 12:01:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:01:17.870249 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc784112-10ec-41af-8542-05bf027aeae1 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:17.870551 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc784112-10ec-41af-8542-05bf027aeae1 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:17.870838 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc784112-10ec-41af-8542-05bf027aeae1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:17.901442 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-dc784112-10ec-41af-8542-05bf027aeae1 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:01:17.902671 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-dc784112-10ec-41af-8542-05bf027aeae1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:01:17.987441 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 105/210] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 25 12:01:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 12:01:18.012012 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c253b676-c2b2-4518-9765-2810515f1830 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:18.071817 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9818bfd8-3449-494b-b1ec-c3e580f4a86e None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:18.072136 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9818bfd8-3449-494b-b1ec-c3e580f4a86e None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:18.072387 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9818bfd8-3449-494b-b1ec-c3e580f4a86e None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:18.073772 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 106/211] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 25 12:01:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:01:18.078663 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fe87b9-4105-455a-8469-b0213751f46d None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:18.078953 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fe87b9-4105-455a-8469-b0213751f46d None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:18.079236 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fe87b9-4105-455a-8469-b0213751f46d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:18.087054 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:18.100380 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-98fe87b9-4105-455a-8469-b0213751f46d None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:01:18.101582 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-98fe87b9-4105-455a-8469-b0213751f46d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:01:18.135651 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c253b676-c2b2-4518-9765-2810515f1830 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:18.175133 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 107/212] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 25 12:01:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 12:01:18.180834 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9386afe3-6ead-4fbe-9cea-170156e4c5a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:18.181833 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9386afe3-6ead-4fbe-9cea-170156e4c5a0 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:18.182276 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9386afe3-6ead-4fbe-9cea-170156e4c5a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:18.182723 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9386afe3-6ead-4fbe-9cea-170156e4c5a0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:18.194973 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9386afe3-6ead-4fbe-9cea-170156e4c5a0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 12:01:18.202665 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fe87b9-4105-455a-8469-b0213751f46d None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:18.203302 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-9386afe3-6ead-4fbe-9cea-170156e4c5a0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 25 12:01:18.203302 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:01:18.203302 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:01:18.203302 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:01:18.203302 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:01:18.203302 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:01:18.203302 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:01:18.203302 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:01:18.203302 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:01:18.203302 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:01:18.203302 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:01:18.203302 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:01:18.203302 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 12:01:18.203302 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 12:01:18.203302 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 12:01:18.203302 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 12:01:18.204173 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 12:01:18.204173 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 12:01:18.204173 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 12:01:18.204173 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 12:01:18.204173 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:01:18.204173 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:01:18.204173 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:01:18.204173 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:01:18.204173 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:01:18.204173 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:01:18.204173 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:01:18.204173 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jul 25 12:01:18.204173 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:01:18.204173 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:01:18.204173 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:01:18.204173 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:01:18.204973 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:01:18.204973 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:01:18.204973 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:01:18.204973 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:01:18.204973 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 12:01:18.204973 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 12:01:18.204973 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 12:01:18.204973 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 12:01:18.204973 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 12:01:18.204973 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 12:01:18.204973 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 25 12:01:18.204973 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jul 25 12:01:18.206997 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 106/213] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Jul 25 12:01:18 2022] GET /identity/v3/roles/member => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:01:18.214628 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9386afe3-6ead-4fbe-9cea-170156e4c5a0 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:18.232510 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:18.332214 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9386afe3-6ead-4fbe-9cea-170156e4c5a0 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:18.332407 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fe87b9-4105-455a-8469-b0213751f46d None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:18.356925 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:18.370673 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44009e86-2de5-404f-88d4-3b3b0db3c045 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:18.371347 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44009e86-2de5-404f-88d4-3b3b0db3c045 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:18.371673 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44009e86-2de5-404f-88d4-3b3b0db3c045 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:18.371960 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44009e86-2de5-404f-88d4-3b3b0db3c045 None admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:18.372721 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44009e86-2de5-404f-88d4-3b3b0db3c045 None admin] RBAC: Adding query filter params (name=member) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:01:18.373040 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44009e86-2de5-404f-88d4-3b3b0db3c045 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:18.374430 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44009e86-2de5-404f-88d4-3b3b0db3c045 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:18.379223 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107e474d-e47b-4629-80c5-8d7670c99f9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:18.380577 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107e474d-e47b-4629-80c5-8d7670c99f9c None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:18.380871 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107e474d-e47b-4629-80c5-8d7670c99f9c None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:18.381153 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107e474d-e47b-4629-80c5-8d7670c99f9c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:18.382595 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107e474d-e47b-4629-80c5-8d7670c99f9c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:18.383887 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 107/214] 10.222.0.42 () {56 vars in 1101 bytes} [Mon Jul 25 12:01:18 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 12:01:18.384070 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107e474d-e47b-4629-80c5-8d7670c99f9c None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:18.385807 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 108/215] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jul 25 12:01:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:01:18.391119 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44009e86-2de5-404f-88d4-3b3b0db3c045 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:18.392851 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107e474d-e47b-4629-80c5-8d7670c99f9c None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:18.503806 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:18.519004 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44009e86-2de5-404f-88d4-3b3b0db3c045 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:18.524134 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107e474d-e47b-4629-80c5-8d7670c99f9c None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:18.559762 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d743c0a-083a-473f-96eb-943460286913 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:18.560630 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d743c0a-083a-473f-96eb-943460286913 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:18.560920 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d743c0a-083a-473f-96eb-943460286913 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:18.561199 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d743c0a-083a-473f-96eb-943460286913 None admin] PATH_INFO: `/v3/users/e43b588aafe24429b8184e1aa1b31dda` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:18.571305 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428853a6-2feb-4fd1-8c0a-ccd5286d5696 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:18.572199 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428853a6-2feb-4fd1-8c0a-ccd5286d5696 None admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:18.572759 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428853a6-2feb-4fd1-8c0a-ccd5286d5696 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:18.573551 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428853a6-2feb-4fd1-8c0a-ccd5286d5696 None admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:18.574966 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428853a6-2feb-4fd1-8c0a-ccd5286d5696 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:18.575838 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d743c0a-083a-473f-96eb-943460286913 None admin] RBAC: Authorizing `identity:get_user(user_id=e43b588aafe24429b8184e1aa1b31dda)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:18.577000 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428853a6-2feb-4fd1-8c0a-ccd5286d5696 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:18.577479 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d743c0a-083a-473f-96eb-943460286913 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:18.579253 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 108/216] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Jul 25 12:01:18 2022] GET /identity/v3/users/e43b588aafe24429b8184e1aa1b31dda => generated 297 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 12:01:18.585324 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d743c0a-083a-473f-96eb-943460286913 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:18.605316 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 109/217] 10.222.0.42 () {60 vars in 1132 bytes} [Mon Jul 25 12:01:18 2022] POST /identity/v3/users => generated 297 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:01:18.634675 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:18.694886 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d743c0a-083a-473f-96eb-943460286913 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:18.731271 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa50c7c-07a3-44e1-baee-ef5cd84b5f6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:18.731974 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa50c7c-07a3-44e1-baee-ef5cd84b5f6e None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:18.732261 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa50c7c-07a3-44e1-baee-ef5cd84b5f6e None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:18.732537 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa50c7c-07a3-44e1-baee-ef5cd84b5f6e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:18.808197 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa50c7c-07a3-44e1-baee-ef5cd84b5f6e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:18.810022 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa50c7c-07a3-44e1-baee-ef5cd84b5f6e None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:18.858052 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:18.919374 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 109/218] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Jul 25 12:01:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 12:01:18.925047 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428853a6-2feb-4fd1-8c0a-ccd5286d5696 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:19.038138 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428853a6-2feb-4fd1-8c0a-ccd5286d5696 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:19.075972 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c81558-c86f-4627-a9fc-2784a3fdb7fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:19.076852 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c81558-c86f-4627-a9fc-2784a3fdb7fb None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:19.077140 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c81558-c86f-4627-a9fc-2784a3fdb7fb None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:19.077417 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c81558-c86f-4627-a9fc-2784a3fdb7fb None admin] PATH_INFO: `/v3/projects/c980d1440d8f49049c8140fffd0d1e15` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:19.078620 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c81558-c86f-4627-a9fc-2784a3fdb7fb None admin] RBAC: Authorizing `identity:get_project(project_id=c980d1440d8f49049c8140fffd0d1e15)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:19.080176 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c81558-c86f-4627-a9fc-2784a3fdb7fb None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:19.082197 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 110/219] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Jul 25 12:01:18 2022] GET /identity/v3/projects/c980d1440d8f49049c8140fffd0d1e15 => generated 302 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:01:19.088211 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa50c7c-07a3-44e1-baee-ef5cd84b5f6e None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:19.204591 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa50c7c-07a3-44e1-baee-ef5cd84b5f6e None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:19.272306 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e97166-7166-48b1-ac1f-6d0353f8e21f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:19.272984 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e97166-7166-48b1-ac1f-6d0353f8e21f None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:19.273274 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e97166-7166-48b1-ac1f-6d0353f8e21f None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:19.273561 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e97166-7166-48b1-ac1f-6d0353f8e21f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:19.274414 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e97166-7166-48b1-ac1f-6d0353f8e21f None admin] RBAC: Adding query filter params (role.id=472313ae69174578824df0bed6390744, scope.project.id=c980d1440d8f49049c8140fffd0d1e15, user.id=e43b588aafe24429b8184e1aa1b31dda) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:01:19.274735 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e97166-7166-48b1-ac1f-6d0353f8e21f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:19.276217 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e97166-7166-48b1-ac1f-6d0353f8e21f None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:19.286225 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 110/220] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Jul 25 12:01:19 2022] GET /identity/v3/role_assignments?user.id=e43b588aafe24429b8184e1aa1b31dda&scope.project.id=c980d1440d8f49049c8140fffd0d1e15&role.id=472313ae69174578824df0bed6390744 => generated 259 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:01:19.430584 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fd0af5-695d-454e-b297-6c7437763976 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:19.430879 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fd0af5-695d-454e-b297-6c7437763976 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:19.431161 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fd0af5-695d-454e-b297-6c7437763976 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:19.432579 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 111/221] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 25 12:01:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:01:19.446515 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99461c8d-3d3e-42f4-a8a9-1c52b00d2b95 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:19.446820 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99461c8d-3d3e-42f4-a8a9-1c52b00d2b95 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:19.447090 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99461c8d-3d3e-42f4-a8a9-1c52b00d2b95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:19.468919 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-99461c8d-3d3e-42f4-a8a9-1c52b00d2b95 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:01:19.470696 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-99461c8d-3d3e-42f4-a8a9-1c52b00d2b95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:01:19.476108 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:19.521848 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a14a31b-90fc-488c-85b2-9b536dc0da52 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:19.522153 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a14a31b-90fc-488c-85b2-9b536dc0da52 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:19.522413 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a14a31b-90fc-488c-85b2-9b536dc0da52 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:19.524068 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 112/222] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 25 12:01:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:01:19.537838 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47a0bae-20d8-40ca-83e7-233a1132e130 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:19.538576 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47a0bae-20d8-40ca-83e7-233a1132e130 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:19.539089 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47a0bae-20d8-40ca-83e7-233a1132e130 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:19.565778 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-e47a0bae-20d8-40ca-83e7-233a1132e130 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:01:19.567333 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-e47a0bae-20d8-40ca-83e7-233a1132e130 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:01:19.573795 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 111/223] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 25 12:01:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 12:01:19.578153 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:19.622656 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99461c8d-3d3e-42f4-a8a9-1c52b00d2b95 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:19.678208 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 113/224] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 25 12:01:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 25 12:01:19.721692 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47a0bae-20d8-40ca-83e7-233a1132e130 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:19.737555 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99461c8d-3d3e-42f4-a8a9-1c52b00d2b95 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:19.777453 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477c9024-828b-44b0-ae07-313ba4f16940 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:19.778717 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477c9024-828b-44b0-ae07-313ba4f16940 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:19.779013 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477c9024-828b-44b0-ae07-313ba4f16940 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:19.779289 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477c9024-828b-44b0-ae07-313ba4f16940 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:19.787844 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:19.793170 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477c9024-828b-44b0-ae07-313ba4f16940 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 12:01:19.804271 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-477c9024-828b-44b0-ae07-313ba4f16940 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 12:01:19.804271 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:01:19.804271 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:01:19.804271 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:01:19.804271 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:01:19.804271 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:01:19.804271 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:01:19.804271 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:01:19.804271 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:01:19.804271 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:01:19.804271 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:01:19.804271 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:01:19.804271 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 12:01:19.804271 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 12:01:19.804271 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 12:01:19.804271 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 12:01:19.805158 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 12:01:19.805158 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 12:01:19.805158 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 12:01:19.805158 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 12:01:19.805158 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:01:19.805158 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:01:19.805158 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:01:19.805158 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:01:19.805158 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:01:19.805158 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:01:19.805158 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:01:19.805158 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jul 25 12:01:19.805158 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:01:19.805158 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:01:19.805158 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:01:19.805158 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:01:19.805931 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:01:19.805931 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:01:19.805931 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:01:19.805931 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:01:19.805931 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 12:01:19.805931 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 12:01:19.805931 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 12:01:19.805931 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 12:01:19.805931 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 12:01:19.805931 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 12:01:19.805931 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 12:01:19.805931 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jul 25 12:01:19.805931 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 112/225] 10.222.0.42 () {56 vars in 1096 bytes} [Mon Jul 25 12:01:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:01:19.814879 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477c9024-828b-44b0-ae07-313ba4f16940 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:19.922841 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47a0bae-20d8-40ca-83e7-233a1132e130 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:19.961599 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aef353b-b4bc-4437-8232-c73f43adbdc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:19.962425 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aef353b-b4bc-4437-8232-c73f43adbdc0 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:19.962712 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aef353b-b4bc-4437-8232-c73f43adbdc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:19.963134 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aef353b-b4bc-4437-8232-c73f43adbdc0 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:19.974885 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aef353b-b4bc-4437-8232-c73f43adbdc0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 12:01:19.983967 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-8aef353b-b4bc-4437-8232-c73f43adbdc0 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 12:01:19.983967 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:01:19.983967 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:01:19.983967 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:01:19.983967 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:01:19.983967 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:01:19.983967 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:01:19.983967 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:01:19.983967 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:01:19.983967 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:01:19.983967 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:01:19.983967 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:01:19.983967 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 12:01:19.983967 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 12:01:19.983967 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 12:01:19.983967 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 12:01:19.984885 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 12:01:19.984885 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 12:01:19.984885 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 12:01:19.984885 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 12:01:19.984885 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:01:19.984885 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:01:19.984885 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:01:19.984885 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:01:19.984885 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:01:19.984885 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:01:19.984885 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:01:19.984885 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 25 12:01:19.984885 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:01:19.984885 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:01:19.984885 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:01:19.984885 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:01:19.986473 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:01:19.986473 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:01:19.986473 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:01:19.986473 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:01:19.986473 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 12:01:19.986473 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 12:01:19.986473 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 12:01:19.986473 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 12:01:19.986473 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 12:01:19.986473 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 12:01:19.986473 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 12:01:19.986473 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 25 12:01:19.986473 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 114/226] 10.222.0.42 () {56 vars in 1114 bytes} [Mon Jul 25 12:01:19 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 12:01:19.998128 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aef353b-b4bc-4437-8232-c73f43adbdc0 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:20.002567 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:20.014994 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477c9024-828b-44b0-ae07-313ba4f16940 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:20.077471 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ae68c5-9c57-4fc5-b0ef-bbf261470904 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:20.078522 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ae68c5-9c57-4fc5-b0ef-bbf261470904 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:20.080363 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ae68c5-9c57-4fc5-b0ef-bbf261470904 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:20.081596 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ae68c5-9c57-4fc5-b0ef-bbf261470904 None admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:20.082560 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ae68c5-9c57-4fc5-b0ef-bbf261470904 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:01:20.083249 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ae68c5-9c57-4fc5-b0ef-bbf261470904 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:20.086005 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ae68c5-9c57-4fc5-b0ef-bbf261470904 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:20.100309 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 113/227] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Jul 25 12:01:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:01:20.107191 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ae68c5-9c57-4fc5-b0ef-bbf261470904 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:20.151004 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aef353b-b4bc-4437-8232-c73f43adbdc0 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:20.207571 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7cc747-80d3-43f5-9d01-63a37550aabd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:20.209760 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7cc747-80d3-43f5-9d01-63a37550aabd None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:20.210883 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7cc747-80d3-43f5-9d01-63a37550aabd None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:20.211387 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7cc747-80d3-43f5-9d01-63a37550aabd None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:20.212522 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7cc747-80d3-43f5-9d01-63a37550aabd None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:01:20.213981 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7cc747-80d3-43f5-9d01-63a37550aabd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:20.215799 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7cc747-80d3-43f5-9d01-63a37550aabd None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:20.216894 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:20.231426 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 115/228] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Jul 25 12:01:19 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 12:01:20.239357 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7cc747-80d3-43f5-9d01-63a37550aabd None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:20.295067 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ae68c5-9c57-4fc5-b0ef-bbf261470904 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:20.345690 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:20.367585 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ea9817-f57f-4d39-8c36-f415ce61342a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:20.367585 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ea9817-f57f-4d39-8c36-f415ce61342a None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:20.367585 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ea9817-f57f-4d39-8c36-f415ce61342a None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:20.367585 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ea9817-f57f-4d39-8c36-f415ce61342a None admin] PATH_INFO: `/v3/users/6f9828f755914644aa268493e126bb00` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:20.368178 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ea9817-f57f-4d39-8c36-f415ce61342a None admin] RBAC: Authorizing `identity:get_user(user_id=6f9828f755914644aa268493e126bb00)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:20.368178 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ea9817-f57f-4d39-8c36-f415ce61342a None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:20.368178 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 114/229] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Jul 25 12:01:20 2022] GET /identity/v3/users/6f9828f755914644aa268493e126bb00 => generated 283 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:01:20.372870 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ea9817-f57f-4d39-8c36-f415ce61342a None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:20.445496 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7cc747-80d3-43f5-9d01-63a37550aabd None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:20.477031 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ea9817-f57f-4d39-8c36-f415ce61342a None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:20.512139 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2328f572-bbcd-4079-82db-91120a885f28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:20.512944 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2328f572-bbcd-4079-82db-91120a885f28 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:20.513243 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2328f572-bbcd-4079-82db-91120a885f28 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:20.513528 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2328f572-bbcd-4079-82db-91120a885f28 None admin] PATH_INFO: `/v3/projects/c980d1440d8f49049c8140fffd0d1e15` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:20.514751 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2328f572-bbcd-4079-82db-91120a885f28 None admin] RBAC: Authorizing `identity:get_project(project_id=c980d1440d8f49049c8140fffd0d1e15)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:20.516253 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2328f572-bbcd-4079-82db-91120a885f28 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:20.518503 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 115/230] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Jul 25 12:01:20 2022] GET /identity/v3/projects/c980d1440d8f49049c8140fffd0d1e15 => generated 302 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 12:01:20.519010 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:20.524064 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2328f572-bbcd-4079-82db-91120a885f28 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:20.548839 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0817daec-e3f7-4248-b0d2-e54401bd15c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:20.549733 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0817daec-e3f7-4248-b0d2-e54401bd15c3 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:20.550016 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0817daec-e3f7-4248-b0d2-e54401bd15c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:20.550294 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0817daec-e3f7-4248-b0d2-e54401bd15c3 None admin] PATH_INFO: `/v3/users/6f9828f755914644aa268493e126bb00` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:20.552820 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0817daec-e3f7-4248-b0d2-e54401bd15c3 None admin] RBAC: Authorizing `identity:get_user(user_id=6f9828f755914644aa268493e126bb00)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:20.554300 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0817daec-e3f7-4248-b0d2-e54401bd15c3 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:20.556297 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 116/231] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Jul 25 12:01:20 2022] GET /identity/v3/users/6f9828f755914644aa268493e126bb00 => generated 283 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:01:20.564486 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0817daec-e3f7-4248-b0d2-e54401bd15c3 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:20.626276 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2328f572-bbcd-4079-82db-91120a885f28 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:20.661198 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304cb2c9-cd29-41d5-a60f-9fe76f8ac5da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:20.662210 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304cb2c9-cd29-41d5-a60f-9fe76f8ac5da None admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:20.662569 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304cb2c9-cd29-41d5-a60f-9fe76f8ac5da None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:20.662924 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304cb2c9-cd29-41d5-a60f-9fe76f8ac5da None admin] PATH_INFO: `/v3/projects/c980d1440d8f49049c8140fffd0d1e15/users/6f9828f755914644aa268493e126bb00/roles/a7d20885b6d94928a34b3cc5b3a00006` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:20.669773 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304cb2c9-cd29-41d5-a60f-9fe76f8ac5da None admin] RBAC: Authorizing `identity:create_grant(project_id=c980d1440d8f49049c8140fffd0d1e15, user_id=6f9828f755914644aa268493e126bb00, role_id=a7d20885b6d94928a34b3cc5b3a00006)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:20.671407 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304cb2c9-cd29-41d5-a60f-9fe76f8ac5da None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:20.696198 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 116/232] 10.222.0.42 () {58 vars in 1439 bytes} [Mon Jul 25 12:01:20 2022] PUT /identity/v3/projects/c980d1440d8f49049c8140fffd0d1e15/users/6f9828f755914644aa268493e126bb00/roles/a7d20885b6d94928a34b3cc5b3a00006 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:01:20.737484 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0817daec-e3f7-4248-b0d2-e54401bd15c3 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:20.780627 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:20.786985 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61afce3-5a55-4fa5-bb10-94598fb5f9fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:20.787964 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61afce3-5a55-4fa5-bb10-94598fb5f9fa None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:20.789744 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61afce3-5a55-4fa5-bb10-94598fb5f9fa None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:20.790289 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61afce3-5a55-4fa5-bb10-94598fb5f9fa None admin] PATH_INFO: `/v3/projects/c980d1440d8f49049c8140fffd0d1e15` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:20.791741 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61afce3-5a55-4fa5-bb10-94598fb5f9fa None admin] RBAC: Authorizing `identity:get_project(project_id=c980d1440d8f49049c8140fffd0d1e15)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:20.793417 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61afce3-5a55-4fa5-bb10-94598fb5f9fa None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:20.795602 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 117/233] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Jul 25 12:01:20 2022] GET /identity/v3/projects/c980d1440d8f49049c8140fffd0d1e15 => generated 302 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:01:20.801957 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304cb2c9-cd29-41d5-a60f-9fe76f8ac5da None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:20.868048 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:20.936642 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304cb2c9-cd29-41d5-a60f-9fe76f8ac5da None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:20.975391 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec00a8e-4621-4fb1-902a-ed20b1a2df07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:20.976335 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec00a8e-4621-4fb1-902a-ed20b1a2df07 None admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:20.976637 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec00a8e-4621-4fb1-902a-ed20b1a2df07 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:20.976906 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec00a8e-4621-4fb1-902a-ed20b1a2df07 None admin] PATH_INFO: `/v3/projects/c980d1440d8f49049c8140fffd0d1e15/users/6f9828f755914644aa268493e126bb00/roles/dea3ef72822043f3baf3bcc1be7c318f` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:20.980066 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec00a8e-4621-4fb1-902a-ed20b1a2df07 None admin] RBAC: Authorizing `identity:create_grant(project_id=c980d1440d8f49049c8140fffd0d1e15, user_id=6f9828f755914644aa268493e126bb00, role_id=dea3ef72822043f3baf3bcc1be7c318f)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:20.980723 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec00a8e-4621-4fb1-902a-ed20b1a2df07 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:20.997397 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 117/234] 10.222.0.42 () {58 vars in 1439 bytes} [Mon Jul 25 12:01:20 2022] PUT /identity/v3/projects/c980d1440d8f49049c8140fffd0d1e15/users/6f9828f755914644aa268493e126bb00/roles/dea3ef72822043f3baf3bcc1be7c318f => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:01:21.276985 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1b70a6-b73c-426b-bbd0-605842425411 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:21.277537 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1b70a6-b73c-426b-bbd0-605842425411 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:21.277822 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1b70a6-b73c-426b-bbd0-605842425411 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:21.279598 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 118/235] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 25 12:01:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:01:21.297665 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff71b77-d0e4-4dee-8188-12d5128600f4 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:21.298153 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff71b77-d0e4-4dee-8188-12d5128600f4 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:21.298790 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff71b77-d0e4-4dee-8188-12d5128600f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:21.321857 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-3ff71b77-d0e4-4dee-8188-12d5128600f4 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:01:21.323059 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-3ff71b77-d0e4-4dee-8188-12d5128600f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:01:21.424182 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 118/236] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 25 12:01:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 12:01:21.464003 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1b70a6-b73c-426b-bbd0-605842425411 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:21.565903 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c346daf3-f668-49eb-9f6f-a269fb46d3c1 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:21.566231 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c346daf3-f668-49eb-9f6f-a269fb46d3c1 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:21.566474 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c346daf3-f668-49eb-9f6f-a269fb46d3c1 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:21.567902 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 119/237] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 25 12:01:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:01:21.573186 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a605060-d92e-4183-a64f-a89c3f2a8826 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:21.573328 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a605060-d92e-4183-a64f-a89c3f2a8826 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:21.573524 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a605060-d92e-4183-a64f-a89c3f2a8826 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:21.602664 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-7a605060-d92e-4183-a64f-a89c3f2a8826 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:01:21.603870 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-7a605060-d92e-4183-a64f-a89c3f2a8826 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:01:21.604802 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1b70a6-b73c-426b-bbd0-605842425411 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:21.649552 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8dab133-89dd-4e6f-9b97-579794d9de67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:21.650701 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dab133-89dd-4e6f-9b97-579794d9de67 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:21.652115 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dab133-89dd-4e6f-9b97-579794d9de67 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:21.653127 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dab133-89dd-4e6f-9b97-579794d9de67 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:21.671016 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:21.676974 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8dab133-89dd-4e6f-9b97-579794d9de67 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 12:01:21.691061 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-c8dab133-89dd-4e6f-9b97-579794d9de67 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 25 12:01:21.691061 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:01:21.691061 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:01:21.691061 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:01:21.691061 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:01:21.691061 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:01:21.691061 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:01:21.691061 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:01:21.691061 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:01:21.691061 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:01:21.691061 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:01:21.691061 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:01:21.691061 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 12:01:21.691061 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 12:01:21.691061 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 12:01:21.691061 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 12:01:21.692009 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 12:01:21.692009 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 12:01:21.692009 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 12:01:21.692009 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 12:01:21.692009 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:01:21.692009 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:01:21.692009 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:01:21.692009 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:01:21.692009 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:01:21.692009 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:01:21.692009 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:01:21.692009 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 25 12:01:21.692009 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:01:21.692009 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:01:21.692009 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:01:21.692009 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:01:21.692845 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:01:21.692845 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:01:21.692845 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:01:21.692845 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:01:21.692845 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 12:01:21.692845 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 12:01:21.692845 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 12:01:21.692845 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 12:01:21.692845 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 12:01:21.692845 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 12:01:21.692845 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 25 12:01:21.692845 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 25 12:01:21.696732 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 119/238] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Jul 25 12:01:21 2022] GET /identity/v3/roles/reader => generated 84 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:01:21.703417 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8dab133-89dd-4e6f-9b97-579794d9de67 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:21.722707 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 120/239] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 25 12:01:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 12:01:21.754580 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a605060-d92e-4183-a64f-a89c3f2a8826 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:21.807913 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:21.879689 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8dab133-89dd-4e6f-9b97-579794d9de67 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:21.916119 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a605060-d92e-4183-a64f-a89c3f2a8826 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:21.922654 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e303c485-fc41-4b1b-a0bd-a143646e6084 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:21.923364 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e303c485-fc41-4b1b-a0bd-a143646e6084 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:21.923680 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e303c485-fc41-4b1b-a0bd-a143646e6084 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:21.923962 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e303c485-fc41-4b1b-a0bd-a143646e6084 None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:21.924719 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e303c485-fc41-4b1b-a0bd-a143646e6084 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:01:21.925588 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e303c485-fc41-4b1b-a0bd-a143646e6084 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:21.927098 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e303c485-fc41-4b1b-a0bd-a143646e6084 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:21.938209 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 120/240] 10.222.0.42 () {56 vars in 1101 bytes} [Mon Jul 25 12:01:21 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:01:21.939761 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263723f4-38b4-4aa4-b454-74abfecfb36c None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:21.940572 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263723f4-38b4-4aa4-b454-74abfecfb36c None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:21.940872 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263723f4-38b4-4aa4-b454-74abfecfb36c None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:21.942405 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 121/241] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 25 12:01:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:01:21.947472 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263723f4-38b4-4aa4-b454-74abfecfb36c None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:21.963908 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f84acc4-8b43-4172-a2e6-6a6ab697e42b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:21.965018 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f84acc4-8b43-4172-a2e6-6a6ab697e42b None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:21.965525 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f84acc4-8b43-4172-a2e6-6a6ab697e42b None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:21.966071 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f84acc4-8b43-4172-a2e6-6a6ab697e42b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:21.967641 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f84acc4-8b43-4172-a2e6-6a6ab697e42b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:21.969425 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f84acc4-8b43-4172-a2e6-6a6ab697e42b None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:21.972317 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 121/242] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jul 25 12:01:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 12:01:21.974025 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1503c97f-2d30-447d-a1f6-ada410d6e80a None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:21.974512 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1503c97f-2d30-447d-a1f6-ada410d6e80a None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:21.974987 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1503c97f-2d30-447d-a1f6-ada410d6e80a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:21.993833 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-1503c97f-2d30-447d-a1f6-ada410d6e80a None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:01:21.995448 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-1503c97f-2d30-447d-a1f6-ada410d6e80a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:01:22.039894 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:22.059822 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263723f4-38b4-4aa4-b454-74abfecfb36c None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:22.068797 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 122/243] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 25 12:01:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 12:01:22.075584 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1503c97f-2d30-447d-a1f6-ada410d6e80a None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:22.096026 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:22.119943 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c87cac-510d-4294-b381-fbd672b265fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:22.121178 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c87cac-510d-4294-b381-fbd672b265fa None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:22.121601 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c87cac-510d-4294-b381-fbd672b265fa None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:22.122046 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c87cac-510d-4294-b381-fbd672b265fa None admin] PATH_INFO: `/v3/users/07ac60e87012498b993322670ab16f61` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:22.137773 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c87cac-510d-4294-b381-fbd672b265fa None admin] RBAC: Authorizing `identity:get_user(user_id=07ac60e87012498b993322670ab16f61)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:22.139586 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c87cac-510d-4294-b381-fbd672b265fa None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:22.141540 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 122/244] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Jul 25 12:01:21 2022] GET /identity/v3/users/07ac60e87012498b993322670ab16f61 => generated 297 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:01:22.142662 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c87cac-510d-4294-b381-fbd672b265fa None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:22.176408 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1503c97f-2d30-447d-a1f6-ada410d6e80a None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:22.212944 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75df5c8-5e5f-492e-b926-d6850b6f8073 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:22.213627 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75df5c8-5e5f-492e-b926-d6850b6f8073 None admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:22.213915 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75df5c8-5e5f-492e-b926-d6850b6f8073 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:22.214195 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75df5c8-5e5f-492e-b926-d6850b6f8073 None admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:22.214911 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75df5c8-5e5f-492e-b926-d6850b6f8073 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:22.216590 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75df5c8-5e5f-492e-b926-d6850b6f8073 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:22.243908 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 123/245] 10.222.0.42 () {60 vars in 1132 bytes} [Mon Jul 25 12:01:22 2022] POST /identity/v3/users => generated 293 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:01:22.244793 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75df5c8-5e5f-492e-b926-d6850b6f8073 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:22.255936 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c87cac-510d-4294-b381-fbd672b265fa None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:22.279811 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:22.300890 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba0638a-49f1-42d6-97e9-4f931926d405 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:22.302362 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba0638a-49f1-42d6-97e9-4f931926d405 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:22.302870 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba0638a-49f1-42d6-97e9-4f931926d405 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:22.303440 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba0638a-49f1-42d6-97e9-4f931926d405 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:22.315891 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba0638a-49f1-42d6-97e9-4f931926d405 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 12:01:22.324549 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-6ba0638a-49f1-42d6-97e9-4f931926d405 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 25 12:01:22.324549 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:01:22.324549 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:01:22.324549 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:01:22.324549 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:01:22.324549 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:01:22.324549 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:01:22.324549 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:01:22.324549 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:01:22.324549 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:01:22.324549 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:01:22.324549 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:01:22.324549 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 12:01:22.324549 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 12:01:22.324549 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 12:01:22.324549 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 12:01:22.325528 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 12:01:22.325528 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 12:01:22.325528 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 12:01:22.325528 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 12:01:22.325528 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:01:22.325528 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:01:22.325528 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:01:22.325528 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:01:22.325528 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:01:22.325528 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:01:22.325528 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:01:22.325528 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 25 12:01:22.325528 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:01:22.325528 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:01:22.325528 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:01:22.325528 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:01:22.326299 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:01:22.326299 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:01:22.326299 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:01:22.326299 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:01:22.326299 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 12:01:22.326299 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 12:01:22.326299 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 12:01:22.326299 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 12:01:22.326299 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 12:01:22.326299 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 12:01:22.326299 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 25 12:01:22.326299 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 25 12:01:22.329040 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 123/246] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Jul 25 12:01:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:01:22.335809 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba0638a-49f1-42d6-97e9-4f931926d405 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:22.362813 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75df5c8-5e5f-492e-b926-d6850b6f8073 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:22.385950 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:22.401738 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68f09ec-7d1d-458c-b326-bc317ccd3b31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:22.402317 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68f09ec-7d1d-458c-b326-bc317ccd3b31 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:22.402607 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68f09ec-7d1d-458c-b326-bc317ccd3b31 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:22.402890 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68f09ec-7d1d-458c-b326-bc317ccd3b31 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:22.454100 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba0638a-49f1-42d6-97e9-4f931926d405 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:22.477617 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68f09ec-7d1d-458c-b326-bc317ccd3b31 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:22.479130 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68f09ec-7d1d-458c-b326-bc317ccd3b31 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:22.493198 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958c4922-b633-40d8-a6fe-0d6ba3c0cb78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:22.494245 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958c4922-b633-40d8-a6fe-0d6ba3c0cb78 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:22.494779 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958c4922-b633-40d8-a6fe-0d6ba3c0cb78 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:22.495251 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958c4922-b633-40d8-a6fe-0d6ba3c0cb78 None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:22.496303 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958c4922-b633-40d8-a6fe-0d6ba3c0cb78 None admin] RBAC: Adding query filter params (name=member) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:01:22.496838 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958c4922-b633-40d8-a6fe-0d6ba3c0cb78 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:22.498550 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958c4922-b633-40d8-a6fe-0d6ba3c0cb78 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:22.513540 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:22.513879 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 124/247] 10.222.0.42 () {56 vars in 1101 bytes} [Mon Jul 25 12:01:22 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 12:01:22.515342 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958c4922-b633-40d8-a6fe-0d6ba3c0cb78 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:22.587573 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 124/248] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Jul 25 12:01:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 12:01:22.593178 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68f09ec-7d1d-458c-b326-bc317ccd3b31 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:22.628395 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958c4922-b633-40d8-a6fe-0d6ba3c0cb78 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:22.630411 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:22.668047 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051d64d9-f366-457a-8e67-ebb895ecb0d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:22.669259 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051d64d9-f366-457a-8e67-ebb895ecb0d1 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:22.669752 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051d64d9-f366-457a-8e67-ebb895ecb0d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:22.670218 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051d64d9-f366-457a-8e67-ebb895ecb0d1 None admin] PATH_INFO: `/v3/users/e43b588aafe24429b8184e1aa1b31dda` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:22.671657 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051d64d9-f366-457a-8e67-ebb895ecb0d1 None admin] RBAC: Authorizing `identity:get_user(user_id=e43b588aafe24429b8184e1aa1b31dda)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:22.673338 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051d64d9-f366-457a-8e67-ebb895ecb0d1 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:22.675186 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 125/249] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Jul 25 12:01:22 2022] GET /identity/v3/users/e43b588aafe24429b8184e1aa1b31dda => generated 297 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:01:22.681520 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051d64d9-f366-457a-8e67-ebb895ecb0d1 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:22.708774 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68f09ec-7d1d-458c-b326-bc317ccd3b31 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:22.746083 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:22.754396 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5759d8c1-884d-4f18-b231-19649458bda4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:22.755293 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5759d8c1-884d-4f18-b231-19649458bda4 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:22.755854 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5759d8c1-884d-4f18-b231-19649458bda4 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:22.756147 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5759d8c1-884d-4f18-b231-19649458bda4 None admin] PATH_INFO: `/v3/projects/c980d1440d8f49049c8140fffd0d1e15` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:22.758071 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5759d8c1-884d-4f18-b231-19649458bda4 None admin] RBAC: Authorizing `identity:get_project(project_id=c980d1440d8f49049c8140fffd0d1e15)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:22.759571 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5759d8c1-884d-4f18-b231-19649458bda4 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:22.762493 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 125/250] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Jul 25 12:01:22 2022] GET /identity/v3/projects/c980d1440d8f49049c8140fffd0d1e15 => generated 302 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 12:01:22.768520 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5759d8c1-884d-4f18-b231-19649458bda4 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:22.817001 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051d64d9-f366-457a-8e67-ebb895ecb0d1 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:22.874901 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488bb39a-3814-4a4d-8771-af11944dc9e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:22.875899 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488bb39a-3814-4a4d-8771-af11944dc9e5 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:22.876190 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488bb39a-3814-4a4d-8771-af11944dc9e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:22.876509 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488bb39a-3814-4a4d-8771-af11944dc9e5 None admin] PATH_INFO: `/v3/projects/c980d1440d8f49049c8140fffd0d1e15` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:22.877742 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488bb39a-3814-4a4d-8771-af11944dc9e5 None admin] RBAC: Authorizing `identity:get_project(project_id=c980d1440d8f49049c8140fffd0d1e15)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:22.879202 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488bb39a-3814-4a4d-8771-af11944dc9e5 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:22.885683 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 126/251] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Jul 25 12:01:22 2022] GET /identity/v3/projects/c980d1440d8f49049c8140fffd0d1e15 => generated 302 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 12:01:22.892819 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488bb39a-3814-4a4d-8771-af11944dc9e5 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:22.921991 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5759d8c1-884d-4f18-b231-19649458bda4 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:22.942121 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:22.963400 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e721132-da27-42d4-b5c4-4e081983b9bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:22.964462 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e721132-da27-42d4-b5c4-4e081983b9bd None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:22.964979 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e721132-da27-42d4-b5c4-4e081983b9bd None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:22.965464 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e721132-da27-42d4-b5c4-4e081983b9bd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:22.966507 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e721132-da27-42d4-b5c4-4e081983b9bd None admin] RBAC: Adding query filter params (role.id=716f62e98b0c4df69e1441697434334c, scope.project.id=c980d1440d8f49049c8140fffd0d1e15, user.id=07ac60e87012498b993322670ab16f61) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:01:22.967046 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e721132-da27-42d4-b5c4-4e081983b9bd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:22.968733 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e721132-da27-42d4-b5c4-4e081983b9bd None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:22.975456 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 126/252] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Jul 25 12:01:22 2022] GET /identity/v3/role_assignments?user.id=07ac60e87012498b993322670ab16f61&scope.project.id=c980d1440d8f49049c8140fffd0d1e15&role.id=716f62e98b0c4df69e1441697434334c => generated 259 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:01:23.022161 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488bb39a-3814-4a4d-8771-af11944dc9e5 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:23.065573 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11e039c-103e-436e-a736-09acbc3f5c98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:23.066499 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11e039c-103e-436e-a736-09acbc3f5c98 None admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:23.066784 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11e039c-103e-436e-a736-09acbc3f5c98 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:23.067065 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11e039c-103e-436e-a736-09acbc3f5c98 None admin] PATH_INFO: `/v3/projects/c980d1440d8f49049c8140fffd0d1e15/users/e43b588aafe24429b8184e1aa1b31dda/roles/472313ae69174578824df0bed6390744` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:23.069866 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11e039c-103e-436e-a736-09acbc3f5c98 None admin] RBAC: Authorizing `identity:create_grant(project_id=c980d1440d8f49049c8140fffd0d1e15, user_id=e43b588aafe24429b8184e1aa1b31dda, role_id=472313ae69174578824df0bed6390744)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:23.071355 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11e039c-103e-436e-a736-09acbc3f5c98 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:23.092118 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 127/253] 10.222.0.42 () {58 vars in 1439 bytes} [Mon Jul 25 12:01:22 2022] PUT /identity/v3/projects/c980d1440d8f49049c8140fffd0d1e15/users/e43b588aafe24429b8184e1aa1b31dda/roles/472313ae69174578824df0bed6390744 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 12:01:23.347843 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafdce9d-4d10-47fa-ae13-0e73f4fb56e6 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:23.348234 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafdce9d-4d10-47fa-ae13-0e73f4fb56e6 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:23.348440 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafdce9d-4d10-47fa-ae13-0e73f4fb56e6 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:23.349798 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 127/254] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 25 12:01:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:01:23.355290 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a344381-2207-4bd9-8bd4-0f8117444059 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:23.355758 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a344381-2207-4bd9-8bd4-0f8117444059 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:23.355956 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a344381-2207-4bd9-8bd4-0f8117444059 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:23.374631 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1fcd04-7782-4c42-a4db-69cd8212e602 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:23.375121 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1fcd04-7782-4c42-a4db-69cd8212e602 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:23.375193 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1fcd04-7782-4c42-a4db-69cd8212e602 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:23.377221 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 128/255] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 25 12:01:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 12:01:23.384956 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2dd28e-8df6-4009-8bb3-e1841e2266c6 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:23.385252 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2dd28e-8df6-4009-8bb3-e1841e2266c6 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:23.385539 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2dd28e-8df6-4009-8bb3-e1841e2266c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:23.389243 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-8a344381-2207-4bd9-8bd4-0f8117444059 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:01:23.390457 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-8a344381-2207-4bd9-8bd4-0f8117444059 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:01:23.421633 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-2e2dd28e-8df6-4009-8bb3-e1841e2266c6 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:01:23.423137 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-2e2dd28e-8df6-4009-8bb3-e1841e2266c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:01:23.528183 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 128/256] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 25 12:01:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 12:01:23.538946 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:23.562245 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 129/257] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 25 12:01:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 12:01:23.564485 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a344381-2207-4bd9-8bd4-0f8117444059 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:23.599416 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2dd28e-8df6-4009-8bb3-e1841e2266c6 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:23.604867 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:23.716422 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a344381-2207-4bd9-8bd4-0f8117444059 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:23.780048 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f138ace0-c24a-4761-927f-bca80f2d3ccf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:23.781329 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f138ace0-c24a-4761-927f-bca80f2d3ccf None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:23.781629 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f138ace0-c24a-4761-927f-bca80f2d3ccf None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:23.781916 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f138ace0-c24a-4761-927f-bca80f2d3ccf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:23.794602 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f138ace0-c24a-4761-927f-bca80f2d3ccf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 12:01:23.802656 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-f138ace0-c24a-4761-927f-bca80f2d3ccf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 12:01:23.802656 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:01:23.802656 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:01:23.802656 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:01:23.802656 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:01:23.802656 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:01:23.802656 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:01:23.802656 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:01:23.802656 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:01:23.802656 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:01:23.802656 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:01:23.802656 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:01:23.802656 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 12:01:23.802656 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 12:01:23.802656 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 12:01:23.802656 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 12:01:23.804700 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 12:01:23.804700 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 12:01:23.804700 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 12:01:23.804700 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 12:01:23.804700 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:01:23.804700 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:01:23.804700 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:01:23.804700 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:01:23.804700 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:01:23.804700 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:01:23.804700 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:01:23.804700 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 25 12:01:23.804700 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:01:23.804700 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:01:23.804700 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:01:23.804700 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:01:23.806637 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:01:23.806637 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:01:23.806637 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:01:23.806637 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:01:23.806637 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 12:01:23.806637 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 12:01:23.806637 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 12:01:23.806637 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 12:01:23.806637 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 12:01:23.806637 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 12:01:23.806637 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 12:01:23.806637 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 25 12:01:23.806637 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 129/258] 10.222.0.42 () {56 vars in 1096 bytes} [Mon Jul 25 12:01:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:01:23.812314 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f138ace0-c24a-4761-927f-bca80f2d3ccf None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:23.942446 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2dd28e-8df6-4009-8bb3-e1841e2266c6 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:24.019410 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:24.033238 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd66b07d-96d1-490a-8eab-e43800ccc097 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:24.034066 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd66b07d-96d1-490a-8eab-e43800ccc097 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:24.034348 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd66b07d-96d1-490a-8eab-e43800ccc097 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:24.034626 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd66b07d-96d1-490a-8eab-e43800ccc097 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:24.052445 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f138ace0-c24a-4761-927f-bca80f2d3ccf None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:24.052852 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd66b07d-96d1-490a-8eab-e43800ccc097 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 12:01:24.069695 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-dd66b07d-96d1-490a-8eab-e43800ccc097 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 12:01:24.069695 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:01:24.069695 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:01:24.069695 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:01:24.069695 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:01:24.069695 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:01:24.069695 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:01:24.069695 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:01:24.069695 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:01:24.069695 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:01:24.069695 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:01:24.069695 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:01:24.069695 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 12:01:24.069695 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 12:01:24.069695 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 12:01:24.069695 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 12:01:24.070618 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 12:01:24.070618 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 12:01:24.070618 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 12:01:24.070618 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 12:01:24.070618 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:01:24.070618 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:01:24.070618 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:01:24.070618 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:01:24.070618 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:01:24.070618 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:01:24.070618 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:01:24.070618 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jul 25 12:01:24.070618 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:01:24.070618 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:01:24.070618 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:01:24.070618 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:01:24.071408 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:01:24.071408 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:01:24.071408 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:01:24.071408 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:01:24.071408 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 12:01:24.071408 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 12:01:24.071408 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 12:01:24.071408 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 12:01:24.071408 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 12:01:24.071408 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 12:01:24.071408 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 12:01:24.071408 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jul 25 12:01:24.071408 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 130/259] 10.222.0.42 () {56 vars in 1114 bytes} [Mon Jul 25 12:01:23 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 472 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:01:24.080631 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd66b07d-96d1-490a-8eab-e43800ccc097 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:24.144789 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c32adc-c82a-4a50-bc78-c4448af3a013 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:24.145943 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c32adc-c82a-4a50-bc78-c4448af3a013 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:24.146431 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c32adc-c82a-4a50-bc78-c4448af3a013 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:24.146907 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c32adc-c82a-4a50-bc78-c4448af3a013 None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:24.147887 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c32adc-c82a-4a50-bc78-c4448af3a013 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:01:24.148409 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c32adc-c82a-4a50-bc78-c4448af3a013 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:24.150114 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c32adc-c82a-4a50-bc78-c4448af3a013 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:24.155572 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:24.166412 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 130/260] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Jul 25 12:01:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 357 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 12:01:24.175079 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c32adc-c82a-4a50-bc78-c4448af3a013 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:24.214300 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd66b07d-96d1-490a-8eab-e43800ccc097 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:24.257346 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6cd6b9-b77c-48a3-984d-b55f15bf9a85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:24.258025 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6cd6b9-b77c-48a3-984d-b55f15bf9a85 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:24.258317 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6cd6b9-b77c-48a3-984d-b55f15bf9a85 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:24.258606 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6cd6b9-b77c-48a3-984d-b55f15bf9a85 None admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:24.259365 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6cd6b9-b77c-48a3-984d-b55f15bf9a85 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:01:24.259713 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6cd6b9-b77c-48a3-984d-b55f15bf9a85 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:24.261101 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6cd6b9-b77c-48a3-984d-b55f15bf9a85 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:24.277493 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 131/261] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Jul 25 12:01:24 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:01:24.287152 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6cd6b9-b77c-48a3-984d-b55f15bf9a85 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:24.395630 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6cd6b9-b77c-48a3-984d-b55f15bf9a85 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:24.405635 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c32adc-c82a-4a50-bc78-c4448af3a013 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:24.410612 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:24.441389 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc70f6d-4b68-4351-9a31-272d7b966687 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:24.442617 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc70f6d-4b68-4351-9a31-272d7b966687 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:24.443048 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc70f6d-4b68-4351-9a31-272d7b966687 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:24.443475 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc70f6d-4b68-4351-9a31-272d7b966687 None admin] PATH_INFO: `/v3/users/6f9828f755914644aa268493e126bb00` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:24.445094 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc70f6d-4b68-4351-9a31-272d7b966687 None admin] RBAC: Authorizing `identity:get_user(user_id=6f9828f755914644aa268493e126bb00)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:24.446801 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc70f6d-4b68-4351-9a31-272d7b966687 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:24.448935 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 132/262] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Jul 25 12:01:24 2022] GET /identity/v3/users/6f9828f755914644aa268493e126bb00 => generated 283 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:01:24.455227 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc70f6d-4b68-4351-9a31-272d7b966687 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:24.460046 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a6785f-a037-47da-bd47-bd0bc05a834b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:24.461121 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a6785f-a037-47da-bd47-bd0bc05a834b None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:24.462555 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a6785f-a037-47da-bd47-bd0bc05a834b None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:24.462984 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a6785f-a037-47da-bd47-bd0bc05a834b None admin] PATH_INFO: `/v3/users/6f9828f755914644aa268493e126bb00` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:24.474118 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a6785f-a037-47da-bd47-bd0bc05a834b None admin] RBAC: Authorizing `identity:get_user(user_id=6f9828f755914644aa268493e126bb00)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:24.475827 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a6785f-a037-47da-bd47-bd0bc05a834b None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:24.477662 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 131/263] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Jul 25 12:01:24 2022] GET /identity/v3/users/6f9828f755914644aa268493e126bb00 => generated 283 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:01:24.491100 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a6785f-a037-47da-bd47-bd0bc05a834b None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:24.541364 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:24.717111 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a6785f-a037-47da-bd47-bd0bc05a834b None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:24.721652 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc70f6d-4b68-4351-9a31-272d7b966687 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:24.792415 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd0189d-0adf-430d-ad77-917f29db78ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:24.793237 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd0189d-0adf-430d-ad77-917f29db78ed None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:24.793554 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd0189d-0adf-430d-ad77-917f29db78ed None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:24.794067 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd0189d-0adf-430d-ad77-917f29db78ed None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:24.809626 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3beabf1e-7c8a-46e3-84f0-851a45a6247b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:24.810294 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3beabf1e-7c8a-46e3-84f0-851a45a6247b None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:24.810585 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3beabf1e-7c8a-46e3-84f0-851a45a6247b None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:24.810874 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3beabf1e-7c8a-46e3-84f0-851a45a6247b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:24.896864 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd0189d-0adf-430d-ad77-917f29db78ed None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:24.898346 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd0189d-0adf-430d-ad77-917f29db78ed None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:24.929958 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:24.972909 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:25.010472 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3beabf1e-7c8a-46e3-84f0-851a45a6247b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:25.028072 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3beabf1e-7c8a-46e3-84f0-851a45a6247b None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:25.130377 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 132/264] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Jul 25 12:01:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 640 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 25 12:01:25.140548 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd0189d-0adf-430d-ad77-917f29db78ed None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:25.190761 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 133/265] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Jul 25 12:01:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 737 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 25 12:01:25.196279 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3beabf1e-7c8a-46e3-84f0-851a45a6247b None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:25.330686 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:25.362601 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3beabf1e-7c8a-46e3-84f0-851a45a6247b None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:25.366104 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd0189d-0adf-430d-ad77-917f29db78ed None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:25.398223 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:25.425886 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6668a3da-19a9-4c31-8924-9073ac4383db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:25.426770 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6668a3da-19a9-4c31-8924-9073ac4383db None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:25.427056 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6668a3da-19a9-4c31-8924-9073ac4383db None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:25.427333 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6668a3da-19a9-4c31-8924-9073ac4383db None admin] PATH_INFO: `/v3/projects/c980d1440d8f49049c8140fffd0d1e15` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:25.429301 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6668a3da-19a9-4c31-8924-9073ac4383db None admin] RBAC: Authorizing `identity:get_project(project_id=c980d1440d8f49049c8140fffd0d1e15)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:25.430768 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6668a3da-19a9-4c31-8924-9073ac4383db None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:25.433615 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 134/266] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Jul 25 12:01:25 2022] GET /identity/v3/projects/c980d1440d8f49049c8140fffd0d1e15 => generated 302 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:01:25.441271 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6668a3da-19a9-4c31-8924-9073ac4383db None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:25.443894 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892a2dfa-7436-40f2-aed3-e85dbc703d94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:25.444787 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892a2dfa-7436-40f2-aed3-e85dbc703d94 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:25.445071 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892a2dfa-7436-40f2-aed3-e85dbc703d94 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:25.445362 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892a2dfa-7436-40f2-aed3-e85dbc703d94 None admin] PATH_INFO: `/v3/projects/c980d1440d8f49049c8140fffd0d1e15` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:25.448687 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892a2dfa-7436-40f2-aed3-e85dbc703d94 None admin] RBAC: Authorizing `identity:get_project(project_id=c980d1440d8f49049c8140fffd0d1e15)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:25.450168 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892a2dfa-7436-40f2-aed3-e85dbc703d94 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:25.456317 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 133/267] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Jul 25 12:01:25 2022] GET /identity/v3/projects/c980d1440d8f49049c8140fffd0d1e15 => generated 302 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:01:25.462129 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892a2dfa-7436-40f2-aed3-e85dbc703d94 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:25.606622 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892a2dfa-7436-40f2-aed3-e85dbc703d94 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:25.666663 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28db7a7-1806-487d-890b-ff7455e9d1e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:25.667710 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28db7a7-1806-487d-890b-ff7455e9d1e5 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:25.668196 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28db7a7-1806-487d-890b-ff7455e9d1e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:25.668685 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28db7a7-1806-487d-890b-ff7455e9d1e5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:25.669743 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28db7a7-1806-487d-890b-ff7455e9d1e5 None admin] RBAC: Adding query filter params (role.id=a7d20885b6d94928a34b3cc5b3a00006, scope.project.id=c980d1440d8f49049c8140fffd0d1e15, user.id=6f9828f755914644aa268493e126bb00) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:01:25.670260 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28db7a7-1806-487d-890b-ff7455e9d1e5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:25.672084 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28db7a7-1806-487d-890b-ff7455e9d1e5 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:25.677574 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6668a3da-19a9-4c31-8924-9073ac4383db None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:25.686515 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 134/268] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Jul 25 12:01:25 2022] GET /identity/v3/role_assignments?user.id=6f9828f755914644aa268493e126bb00&scope.project.id=c980d1440d8f49049c8140fffd0d1e15&role.id=a7d20885b6d94928a34b3cc5b3a00006 => generated 608 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:01:25.743784 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec61833c-269f-451c-b197-5dc7e3c7e3ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:25.744541 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec61833c-269f-451c-b197-5dc7e3c7e3ec None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:25.744832 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec61833c-269f-451c-b197-5dc7e3c7e3ec None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:25.745117 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec61833c-269f-451c-b197-5dc7e3c7e3ec None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:25.745981 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec61833c-269f-451c-b197-5dc7e3c7e3ec None admin] RBAC: Adding query filter params (role.id=dea3ef72822043f3baf3bcc1be7c318f, scope.project.id=c980d1440d8f49049c8140fffd0d1e15, user.id=6f9828f755914644aa268493e126bb00) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:01:25.746290 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec61833c-269f-451c-b197-5dc7e3c7e3ec None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:25.747776 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec61833c-269f-451c-b197-5dc7e3c7e3ec None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:25.759302 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 135/269] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Jul 25 12:01:25 2022] GET /identity/v3/role_assignments?user.id=6f9828f755914644aa268493e126bb00&scope.project.id=c980d1440d8f49049c8140fffd0d1e15&role.id=dea3ef72822043f3baf3bcc1be7c318f => generated 608 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:01:25.835655 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324968e1-3130-4f66-9b71-7363af64b2cc None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:25.836043 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52170550-c76f-40ed-a472-a058a7552943 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:25.836352 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52170550-c76f-40ed-a472-a058a7552943 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:25.836640 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52170550-c76f-40ed-a472-a058a7552943 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:25.836965 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324968e1-3130-4f66-9b71-7363af64b2cc None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:25.837435 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324968e1-3130-4f66-9b71-7363af64b2cc None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:25.838269 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 136/270] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 25 12:01:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:01:25.839410 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 135/271] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 25 12:01:25 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:01:25.843818 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4a38e6-a0d9-4443-99c8-a584e29a3ed1 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:25.844106 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4a38e6-a0d9-4443-99c8-a584e29a3ed1 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:25.844390 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4a38e6-a0d9-4443-99c8-a584e29a3ed1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:25.846116 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6a7d3e-bae8-4460-928f-a49572ec4771 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:25.846614 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6a7d3e-bae8-4460-928f-a49572ec4771 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:25.847082 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6a7d3e-bae8-4460-928f-a49572ec4771 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:25.875208 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-ba4a38e6-a0d9-4443-99c8-a584e29a3ed1 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:01:25.875208 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-ba4a38e6-a0d9-4443-99c8-a584e29a3ed1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:01:25.878020 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-3e6a7d3e-bae8-4460-928f-a49572ec4771 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:01:25.880603 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-3e6a7d3e-bae8-4460-928f-a49572ec4771 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:01:25.895437 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:25.962223 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:25.994040 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 136/272] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 25 12:01:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 25 12:01:26.036829 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6a7d3e-bae8-4460-928f-a49572ec4771 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:26.042446 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 137/273] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 25 12:01:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 12:01:26.076961 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7277f58-5331-4ccf-ab88-3755f47f571b None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:26.077649 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7277f58-5331-4ccf-ab88-3755f47f571b None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:26.078124 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7277f58-5331-4ccf-ab88-3755f47f571b None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:26.080412 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 138/274] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 25 12:01:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 12:01:26.081055 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7277f58-5331-4ccf-ab88-3755f47f571b None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:26.161893 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6a7d3e-bae8-4460-928f-a49572ec4771 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:26.200207 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fe0ad0-c28e-4f93-b7b4-741711267dde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:26.201013 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fe0ad0-c28e-4f93-b7b4-741711267dde None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:26.201304 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fe0ad0-c28e-4f93-b7b4-741711267dde None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:26.201586 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fe0ad0-c28e-4f93-b7b4-741711267dde None admin] PATH_INFO: `/v3/domains/default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:26.202758 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fe0ad0-c28e-4f93-b7b4-741711267dde None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:26.204253 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fe0ad0-c28e-4f93-b7b4-741711267dde None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:26.207070 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 137/275] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jul 25 12:01:26 2022] GET /identity/v3/domains/default => generated 199 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 12:01:26.208160 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbc20aa-8258-44df-9ec2-011ac682ddf3 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:26.208455 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbc20aa-8258-44df-9ec2-011ac682ddf3 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:26.208736 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbc20aa-8258-44df-9ec2-011ac682ddf3 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:26.209968 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 138/276] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 25 12:01:26 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:01:26.211011 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619e8056-c089-444d-9f9f-422d37f5ba51 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:26.211500 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619e8056-c089-444d-9f9f-422d37f5ba51 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:26.211668 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619e8056-c089-444d-9f9f-422d37f5ba51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:26.218456 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7277f58-5331-4ccf-ab88-3755f47f571b None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:26.228844 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-619e8056-c089-444d-9f9f-422d37f5ba51 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:01:26.230042 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-619e8056-c089-444d-9f9f-422d37f5ba51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:01:26.252691 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8fb841-711a-4405-abfc-d7022b83b4c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:26.253488 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8fb841-711a-4405-abfc-d7022b83b4c8 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:26.253778 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8fb841-711a-4405-abfc-d7022b83b4c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:26.254062 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8fb841-711a-4405-abfc-d7022b83b4c8 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:26.264763 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8fb841-711a-4405-abfc-d7022b83b4c8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 12:01:26.273008 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-1f8fb841-711a-4405-abfc-d7022b83b4c8 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 25 12:01:26.273008 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:01:26.273008 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:01:26.273008 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:01:26.273008 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:01:26.273008 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:01:26.273008 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:01:26.273008 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:01:26.273008 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:01:26.273008 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:01:26.273008 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:01:26.273008 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:01:26.273008 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 12:01:26.273008 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 12:01:26.273008 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 12:01:26.273008 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 12:01:26.273913 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 12:01:26.273913 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 12:01:26.273913 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 12:01:26.273913 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 12:01:26.273913 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:01:26.273913 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:01:26.273913 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:01:26.273913 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:01:26.273913 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:01:26.273913 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:01:26.273913 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:01:26.273913 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jul 25 12:01:26.273913 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:01:26.273913 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:01:26.273913 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:01:26.273913 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:01:26.274801 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:01:26.274801 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:01:26.274801 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:01:26.274801 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:01:26.274801 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 12:01:26.274801 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 12:01:26.274801 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 12:01:26.274801 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 12:01:26.274801 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 12:01:26.274801 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 12:01:26.274801 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 25 12:01:26.274801 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jul 25 12:01:26.274801 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 139/277] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Jul 25 12:01:26 2022] GET /identity/v3/roles/reader => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:01:26.275475 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8fb841-711a-4405-abfc-d7022b83b4c8 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:26.277690 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:26.279576 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:26.299187 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 139/278] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 25 12:01:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 12:01:26.300615 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f303ad77-3881-4c6d-a146-90c51e500036 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:26.300913 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f303ad77-3881-4c6d-a146-90c51e500036 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:26.301198 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f303ad77-3881-4c6d-a146-90c51e500036 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:26.318800 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-f303ad77-3881-4c6d-a146-90c51e500036 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:01:26.320028 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-f303ad77-3881-4c6d-a146-90c51e500036 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:01:26.380781 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8fb841-711a-4405-abfc-d7022b83b4c8 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:26.386361 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 140/279] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 25 12:01:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 12:01:26.387184 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f303ad77-3881-4c6d-a146-90c51e500036 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:26.420393 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8419a45d-c484-4ea6-805c-817a8b23cc3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:26.421070 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8419a45d-c484-4ea6-805c-817a8b23cc3c None admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:26.421357 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8419a45d-c484-4ea6-805c-817a8b23cc3c None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:26.421641 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8419a45d-c484-4ea6-805c-817a8b23cc3c None admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:26.422363 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8419a45d-c484-4ea6-805c-817a8b23cc3c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:26.423874 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8419a45d-c484-4ea6-805c-817a8b23cc3c None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:26.445113 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 140/280] 10.222.0.42 () {60 vars in 1132 bytes} [Mon Jul 25 12:01:26 2022] POST /identity/v3/users => generated 293 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:01:26.446021 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8419a45d-c484-4ea6-805c-817a8b23cc3c None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:26.497770 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f303ad77-3881-4c6d-a146-90c51e500036 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:26.509700 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:26.519598 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:26.539323 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3196148-f61c-4813-9ecd-077dc538c39b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:26.540462 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3196148-f61c-4813-9ecd-077dc538c39b None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:26.540754 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3196148-f61c-4813-9ecd-077dc538c39b None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:26.541034 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3196148-f61c-4813-9ecd-077dc538c39b None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:26.541809 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3196148-f61c-4813-9ecd-077dc538c39b None admin] RBAC: Adding query filter params (name=reader) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:01:26.542120 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3196148-f61c-4813-9ecd-077dc538c39b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:26.543554 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3196148-f61c-4813-9ecd-077dc538c39b None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:26.553721 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 141/281] 10.222.0.42 () {56 vars in 1101 bytes} [Mon Jul 25 12:01:26 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:01:26.559386 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3196148-f61c-4813-9ecd-077dc538c39b None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:26.572779 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8419a45d-c484-4ea6-805c-817a8b23cc3c None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:26.613848 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e81c54-5937-4c3b-be86-3eadba10b584 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:26.614709 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e81c54-5937-4c3b-be86-3eadba10b584 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:26.615053 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e81c54-5937-4c3b-be86-3eadba10b584 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:26.615393 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e81c54-5937-4c3b-be86-3eadba10b584 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:26.627302 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e81c54-5937-4c3b-be86-3eadba10b584 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 12:01:26.635142 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-e9e81c54-5937-4c3b-be86-3eadba10b584 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 25 12:01:26.635142 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:01:26.635142 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:01:26.635142 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:01:26.635142 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:01:26.635142 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:01:26.635142 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:01:26.635142 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:01:26.635142 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:01:26.635142 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:01:26.635142 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:01:26.635142 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:01:26.635142 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 12:01:26.635142 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 12:01:26.635142 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 12:01:26.635142 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 12:01:26.636085 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 12:01:26.636085 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 12:01:26.636085 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 12:01:26.636085 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 12:01:26.636085 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:01:26.636085 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:01:26.636085 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:01:26.636085 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:01:26.636085 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:01:26.636085 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:01:26.636085 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:01:26.636085 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jul 25 12:01:26.636085 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:01:26.636085 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:01:26.636085 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:01:26.636085 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:01:26.636889 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:01:26.636889 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:01:26.636889 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:01:26.636889 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:01:26.636889 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 12:01:26.636889 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 12:01:26.636889 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 12:01:26.636889 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 12:01:26.636889 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 12:01:26.636889 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 12:01:26.636889 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 25 12:01:26.636889 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jul 25 12:01:26.636889 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 141/282] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Jul 25 12:01:26 2022] GET /identity/v3/roles/member => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:01:26.637562 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e81c54-5937-4c3b-be86-3eadba10b584 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:26.689985 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3196148-f61c-4813-9ecd-077dc538c39b None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:26.733693 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e133f7f6-79b0-4f94-8f22-fb8581435e7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:26.734960 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e133f7f6-79b0-4f94-8f22-fb8581435e7d None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:26.735254 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e133f7f6-79b0-4f94-8f22-fb8581435e7d None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:26.735539 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e133f7f6-79b0-4f94-8f22-fb8581435e7d None admin] PATH_INFO: `/v3/roles/member` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:26.749805 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e133f7f6-79b0-4f94-8f22-fb8581435e7d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 12:01:26.758228 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-e133f7f6-79b0-4f94-8f22-fb8581435e7d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 25 12:01:26.758228 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:01:26.758228 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:01:26.758228 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:01:26.758228 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:01:26.758228 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:01:26.758228 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:01:26.758228 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:01:26.758228 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:01:26.758228 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:01:26.758228 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:01:26.758228 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:01:26.758228 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 12:01:26.758228 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 12:01:26.758228 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 12:01:26.758228 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 12:01:26.759098 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 12:01:26.759098 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 12:01:26.759098 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 12:01:26.759098 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 12:01:26.759098 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:01:26.759098 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:01:26.759098 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:01:26.759098 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:01:26.759098 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:01:26.759098 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:01:26.759098 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:01:26.759098 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 25 12:01:26.759098 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:01:26.759098 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:01:26.759098 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:01:26.759098 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:01:26.759950 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:01:26.759950 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:01:26.759950 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:01:26.759950 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:01:26.759950 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 12:01:26.759950 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 12:01:26.759950 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 12:01:26.759950 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 12:01:26.759950 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 12:01:26.759950 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 12:01:26.759950 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 25 12:01:26.759950 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 25 12:01:26.759950 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 142/283] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Jul 25 12:01:26 2022] GET /identity/v3/roles/member => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:01:26.760597 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e133f7f6-79b0-4f94-8f22-fb8581435e7d None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:26.765029 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e81c54-5937-4c3b-be86-3eadba10b584 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:26.791889 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:26.811046 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:26.813518 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590be1f2-d13d-4ad0-875c-478c626dcb4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:26.815088 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590be1f2-d13d-4ad0-875c-478c626dcb4c None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:26.815376 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590be1f2-d13d-4ad0-875c-478c626dcb4c None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:26.815731 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590be1f2-d13d-4ad0-875c-478c626dcb4c None admin] PATH_INFO: `/v3/users/07ac60e87012498b993322670ab16f61` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:26.816980 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590be1f2-d13d-4ad0-875c-478c626dcb4c None admin] RBAC: Authorizing `identity:get_user(user_id=07ac60e87012498b993322670ab16f61)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:26.818457 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590be1f2-d13d-4ad0-875c-478c626dcb4c None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:26.820539 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 142/284] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Jul 25 12:01:26 2022] GET /identity/v3/users/07ac60e87012498b993322670ab16f61 => generated 297 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:01:26.821723 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590be1f2-d13d-4ad0-875c-478c626dcb4c None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:26.907247 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e133f7f6-79b0-4f94-8f22-fb8581435e7d None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:26.943671 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572ab671-894d-4c51-9a52-17910fa27466 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:26.944379 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572ab671-894d-4c51-9a52-17910fa27466 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:26.944673 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572ab671-894d-4c51-9a52-17910fa27466 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:26.944959 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572ab671-894d-4c51-9a52-17910fa27466 None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:26.945724 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572ab671-894d-4c51-9a52-17910fa27466 None admin] RBAC: Adding query filter params (name=member) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:01:26.946040 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572ab671-894d-4c51-9a52-17910fa27466 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:26.947441 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572ab671-894d-4c51-9a52-17910fa27466 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:26.951065 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590be1f2-d13d-4ad0-875c-478c626dcb4c None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:26.956767 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 143/285] 10.222.0.42 () {56 vars in 1101 bytes} [Mon Jul 25 12:01:26 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:01:26.957668 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572ab671-894d-4c51-9a52-17910fa27466 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:26.998452 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397d1c64-642a-43c6-a521-193daba8a685 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:26.999129 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397d1c64-642a-43c6-a521-193daba8a685 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:26.999421 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397d1c64-642a-43c6-a521-193daba8a685 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:26.999740 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397d1c64-642a-43c6-a521-193daba8a685 None admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:27.001085 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397d1c64-642a-43c6-a521-193daba8a685 None admin] RBAC: Adding query filter params (name=member) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:01:27.001384 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397d1c64-642a-43c6-a521-193daba8a685 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:27.003254 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397d1c64-642a-43c6-a521-193daba8a685 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:27.014875 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 143/286] 10.222.0.42 () {56 vars in 1101 bytes} [Mon Jul 25 12:01:26 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:01:27.016039 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397d1c64-642a-43c6-a521-193daba8a685 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:27.072324 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:27.075022 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:27.090936 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572ab671-894d-4c51-9a52-17910fa27466 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:27.125762 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f77c330-e4e9-4947-9c62-173ea2f2848f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:27.126679 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f77c330-e4e9-4947-9c62-173ea2f2848f None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:27.126972 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f77c330-e4e9-4947-9c62-173ea2f2848f None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:27.127258 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f77c330-e4e9-4947-9c62-173ea2f2848f None admin] PATH_INFO: `/v3/projects/c980d1440d8f49049c8140fffd0d1e15` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:27.128520 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f77c330-e4e9-4947-9c62-173ea2f2848f None admin] RBAC: Authorizing `identity:get_project(project_id=c980d1440d8f49049c8140fffd0d1e15)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:27.130087 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f77c330-e4e9-4947-9c62-173ea2f2848f None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:27.132206 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 144/287] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Jul 25 12:01:26 2022] GET /identity/v3/projects/c980d1440d8f49049c8140fffd0d1e15 => generated 302 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:01:27.133110 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f77c330-e4e9-4947-9c62-173ea2f2848f None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:27.144380 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397d1c64-642a-43c6-a521-193daba8a685 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:27.182531 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69072c6-ab09-4923-80f7-a22c482206cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:27.183577 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69072c6-ab09-4923-80f7-a22c482206cc None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:27.183881 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69072c6-ab09-4923-80f7-a22c482206cc None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:27.184170 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69072c6-ab09-4923-80f7-a22c482206cc None admin] PATH_INFO: `/v3/users/e43b588aafe24429b8184e1aa1b31dda` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:27.185755 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69072c6-ab09-4923-80f7-a22c482206cc None admin] RBAC: Authorizing `identity:get_user(user_id=e43b588aafe24429b8184e1aa1b31dda)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:27.187205 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69072c6-ab09-4923-80f7-a22c482206cc None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:27.189151 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 144/288] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Jul 25 12:01:27 2022] GET /identity/v3/users/e43b588aafe24429b8184e1aa1b31dda => generated 297 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:01:27.189897 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69072c6-ab09-4923-80f7-a22c482206cc None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:27.241729 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f77c330-e4e9-4947-9c62-173ea2f2848f None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:27.277260 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7b87ec-c570-40cc-97f6-0e4c42aa1776 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:27.278465 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7b87ec-c570-40cc-97f6-0e4c42aa1776 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:27.278955 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7b87ec-c570-40cc-97f6-0e4c42aa1776 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:27.279437 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7b87ec-c570-40cc-97f6-0e4c42aa1776 None admin] PATH_INFO: `/v3/users/6c8e5f6d7cb346568741bfa75792bb14` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:27.294816 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7b87ec-c570-40cc-97f6-0e4c42aa1776 None admin] RBAC: Authorizing `identity:get_user(user_id=6c8e5f6d7cb346568741bfa75792bb14)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:27.296621 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7b87ec-c570-40cc-97f6-0e4c42aa1776 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:27.299102 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 145/289] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Jul 25 12:01:27 2022] GET /identity/v3/users/6c8e5f6d7cb346568741bfa75792bb14 => generated 293 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:01:27.300253 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7b87ec-c570-40cc-97f6-0e4c42aa1776 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:27.312014 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69072c6-ab09-4923-80f7-a22c482206cc None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:27.320794 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:27.333831 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:27.354150 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ec3b40-3ce5-4f5b-9b96-c46b4824e8bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:27.355183 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ec3b40-3ce5-4f5b-9b96-c46b4824e8bc None admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:27.355477 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ec3b40-3ce5-4f5b-9b96-c46b4824e8bc None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:27.355816 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ec3b40-3ce5-4f5b-9b96-c46b4824e8bc None admin] PATH_INFO: `/v3/projects/c980d1440d8f49049c8140fffd0d1e15/users/07ac60e87012498b993322670ab16f61/roles/716f62e98b0c4df69e1441697434334c` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:27.358058 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ec3b40-3ce5-4f5b-9b96-c46b4824e8bc None admin] RBAC: Authorizing `identity:create_grant(project_id=c980d1440d8f49049c8140fffd0d1e15, user_id=07ac60e87012498b993322670ab16f61, role_id=716f62e98b0c4df69e1441697434334c)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:27.359767 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ec3b40-3ce5-4f5b-9b96-c46b4824e8bc None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:27.376765 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 145/290] 10.222.0.42 () {58 vars in 1439 bytes} [Mon Jul 25 12:01:27 2022] PUT /identity/v3/projects/c980d1440d8f49049c8140fffd0d1e15/users/07ac60e87012498b993322670ab16f61/roles/716f62e98b0c4df69e1441697434334c => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:01:27.377675 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ec3b40-3ce5-4f5b-9b96-c46b4824e8bc None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:27.426708 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7b87ec-c570-40cc-97f6-0e4c42aa1776 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:27.466526 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342030fd-37cc-4373-85f0-4b1f2bf4d173 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:27.468288 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342030fd-37cc-4373-85f0-4b1f2bf4d173 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:27.468721 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342030fd-37cc-4373-85f0-4b1f2bf4d173 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:27.469137 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342030fd-37cc-4373-85f0-4b1f2bf4d173 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:27.523369 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ec3b40-3ce5-4f5b-9b96-c46b4824e8bc None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:27.560243 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342030fd-37cc-4373-85f0-4b1f2bf4d173 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:27.562778 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342030fd-37cc-4373-85f0-4b1f2bf4d173 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:27.568757 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2bbd24-293f-4cfb-9082-023479e85770 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:27.569585 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2bbd24-293f-4cfb-9082-023479e85770 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:27.569988 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2bbd24-293f-4cfb-9082-023479e85770 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:27.570373 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2bbd24-293f-4cfb-9082-023479e85770 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:27.571337 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2bbd24-293f-4cfb-9082-023479e85770 None admin] RBAC: Adding query filter params (role.id=472313ae69174578824df0bed6390744, user.id=6c8e5f6d7cb346568741bfa75792bb14, scope.system=all) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:01:27.571814 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2bbd24-293f-4cfb-9082-023479e85770 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:27.573482 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2bbd24-293f-4cfb-9082-023479e85770 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:27.581720 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 146/291] 10.222.0.42 () {56 vars in 1308 bytes} [Mon Jul 25 12:01:27 2022] GET /identity/v3/role_assignments?user.id=6c8e5f6d7cb346568741bfa75792bb14&scope.system=all&role.id=472313ae69174578824df0bed6390744 => generated 226 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:01:27.588640 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:27.690766 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 146/292] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Jul 25 12:01:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 12:01:27.696072 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2bbd24-293f-4cfb-9082-023479e85770 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:27.764504 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:27.868810 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2bbd24-293f-4cfb-9082-023479e85770 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:27.921156 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5076ceb-ed08-4448-ae98-6e9a8d6ea18d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:27.922093 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5076ceb-ed08-4448-ae98-6e9a8d6ea18d None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:27.922430 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5076ceb-ed08-4448-ae98-6e9a8d6ea18d None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:27.922713 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5076ceb-ed08-4448-ae98-6e9a8d6ea18d None admin] PATH_INFO: `/v3/projects/c980d1440d8f49049c8140fffd0d1e15` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:27.924441 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5076ceb-ed08-4448-ae98-6e9a8d6ea18d None admin] RBAC: Authorizing `identity:get_project(project_id=c980d1440d8f49049c8140fffd0d1e15)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:27.925954 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5076ceb-ed08-4448-ae98-6e9a8d6ea18d None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:27.927693 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 147/293] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Jul 25 12:01:27 2022] GET /identity/v3/projects/c980d1440d8f49049c8140fffd0d1e15 => generated 302 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:01:27.934957 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342030fd-37cc-4373-85f0-4b1f2bf4d173 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:28.065870 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342030fd-37cc-4373-85f0-4b1f2bf4d173 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:28.084710 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:28.110607 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef34ce7a-4078-4125-a98f-01f19d54e174 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:28.111839 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef34ce7a-4078-4125-a98f-01f19d54e174 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:28.112136 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef34ce7a-4078-4125-a98f-01f19d54e174 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:28.112418 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef34ce7a-4078-4125-a98f-01f19d54e174 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:28.113268 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef34ce7a-4078-4125-a98f-01f19d54e174 None admin] RBAC: Adding query filter params (role.id=472313ae69174578824df0bed6390744, scope.project.id=c980d1440d8f49049c8140fffd0d1e15, user.id=e43b588aafe24429b8184e1aa1b31dda) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:01:28.113587 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef34ce7a-4078-4125-a98f-01f19d54e174 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:28.115043 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef34ce7a-4078-4125-a98f-01f19d54e174 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:28.124230 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 147/294] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Jul 25 12:01:27 2022] GET /identity/v3/role_assignments?user.id=e43b588aafe24429b8184e1aa1b31dda&scope.project.id=c980d1440d8f49049c8140fffd0d1e15&role.id=472313ae69174578824df0bed6390744 => generated 608 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:01:28.808395 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5cf14a-5160-421d-a28b-acf21f16d3cc None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:28.809032 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5cf14a-5160-421d-a28b-acf21f16d3cc None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:28.809032 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5cf14a-5160-421d-a28b-acf21f16d3cc None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:28.810615 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 148/295] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 25 12:01:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:01:28.816323 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8333353-d56b-43a9-a96a-76c915a30c2e None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:28.816822 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8333353-d56b-43a9-a96a-76c915a30c2e None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:28.816989 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8333353-d56b-43a9-a96a-76c915a30c2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:28.835518 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-c8333353-d56b-43a9-a96a-76c915a30c2e None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:01:28.837169 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-c8333353-d56b-43a9-a96a-76c915a30c2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:01:28.843431 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15256760-d0a2-431f-80fc-7fb23524a3b5 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:28.844117 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15256760-d0a2-431f-80fc-7fb23524a3b5 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:28.844625 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15256760-d0a2-431f-80fc-7fb23524a3b5 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:28.846317 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 149/296] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 25 12:01:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:01:28.852397 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973b04f0-ccdc-485d-8878-03822cf9a7c8 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:28.852973 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973b04f0-ccdc-485d-8878-03822cf9a7c8 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:28.853770 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973b04f0-ccdc-485d-8878-03822cf9a7c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:28.877430 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-973b04f0-ccdc-485d-8878-03822cf9a7c8 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:01:28.879437 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-973b04f0-ccdc-485d-8878-03822cf9a7c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:01:28.935592 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 148/297] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 25 12:01:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 12:01:28.951647 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8333353-d56b-43a9-a96a-76c915a30c2e None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:28.963788 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 150/298] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 25 12:01:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 12:01:28.987030 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973b04f0-ccdc-485d-8878-03822cf9a7c8 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:29.043789 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:29.067159 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8333353-d56b-43a9-a96a-76c915a30c2e None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:29.081688 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:29.106219 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875ac429-9730-4027-8297-5c5e01b4fffb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:29.107399 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875ac429-9730-4027-8297-5c5e01b4fffb None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:29.107931 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875ac429-9730-4027-8297-5c5e01b4fffb None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:29.108409 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875ac429-9730-4027-8297-5c5e01b4fffb None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:29.120224 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875ac429-9730-4027-8297-5c5e01b4fffb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 12:01:29.121568 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973b04f0-ccdc-485d-8878-03822cf9a7c8 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:29.128693 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-875ac429-9730-4027-8297-5c5e01b4fffb None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 25 12:01:29.128693 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:01:29.128693 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:01:29.128693 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:01:29.128693 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:01:29.128693 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:01:29.128693 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:01:29.128693 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:01:29.128693 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:01:29.128693 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:01:29.128693 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:01:29.128693 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:01:29.128693 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 12:01:29.128693 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 12:01:29.128693 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 12:01:29.128693 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 12:01:29.129567 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 12:01:29.129567 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 12:01:29.129567 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 12:01:29.129567 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 12:01:29.129567 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:01:29.129567 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:01:29.129567 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:01:29.129567 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:01:29.129567 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:01:29.129567 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:01:29.129567 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:01:29.129567 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 25 12:01:29.129567 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:01:29.129567 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:01:29.129567 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:01:29.129567 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:01:29.130378 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:01:29.130378 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:01:29.130378 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:01:29.130378 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:01:29.130378 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 12:01:29.130378 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 12:01:29.130378 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 12:01:29.130378 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 12:01:29.130378 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 12:01:29.130378 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 12:01:29.130378 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 25 12:01:29.130378 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 25 12:01:29.133222 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 149/299] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Jul 25 12:01:28 2022] GET /identity/v3/roles/reader => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 12:01:29.139085 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875ac429-9730-4027-8297-5c5e01b4fffb None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:29.157210 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f49e31-ff95-4108-a956-ae4e7bdad990 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:29.158915 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f49e31-ff95-4108-a956-ae4e7bdad990 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:29.159411 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f49e31-ff95-4108-a956-ae4e7bdad990 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:29.159922 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f49e31-ff95-4108-a956-ae4e7bdad990 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:29.161396 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f49e31-ff95-4108-a956-ae4e7bdad990 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:29.163120 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f49e31-ff95-4108-a956-ae4e7bdad990 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:29.169847 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 151/300] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jul 25 12:01:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 12:01:29.170863 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f49e31-ff95-4108-a956-ae4e7bdad990 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:29.257521 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875ac429-9730-4027-8297-5c5e01b4fffb None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:29.285957 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f49e31-ff95-4108-a956-ae4e7bdad990 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:29.295828 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:29.302676 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2486531f-df18-450b-a429-ac4e1b8f7c90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:29.303679 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2486531f-df18-450b-a429-ac4e1b8f7c90 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:29.304237 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2486531f-df18-450b-a429-ac4e1b8f7c90 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:29.304830 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2486531f-df18-450b-a429-ac4e1b8f7c90 None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:29.305786 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2486531f-df18-450b-a429-ac4e1b8f7c90 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:01:29.306293 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2486531f-df18-450b-a429-ac4e1b8f7c90 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:29.308015 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2486531f-df18-450b-a429-ac4e1b8f7c90 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:29.322360 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 150/301] 10.222.0.42 () {56 vars in 1101 bytes} [Mon Jul 25 12:01:29 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:01:29.323656 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2486531f-df18-450b-a429-ac4e1b8f7c90 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:29.326808 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1875e21e-54a1-4424-8dd0-ca49cdead8bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:29.328300 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1875e21e-54a1-4424-8dd0-ca49cdead8bd None admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:29.328584 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1875e21e-54a1-4424-8dd0-ca49cdead8bd None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:29.328862 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1875e21e-54a1-4424-8dd0-ca49cdead8bd None admin] PATH_INFO: `/v3/groups` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:29.330109 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1875e21e-54a1-4424-8dd0-ca49cdead8bd None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:29.331598 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1875e21e-54a1-4424-8dd0-ca49cdead8bd None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:29.359590 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 152/302] 10.222.0.42 () {60 vars in 1134 bytes} [Mon Jul 25 12:01:29 2022] POST /identity/v3/groups => generated 229 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:01:29.374725 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:29.451260 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2486531f-df18-450b-a429-ac4e1b8f7c90 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:29.490246 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ae6899-d728-4f8e-a8f5-735787d23ee8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:29.491207 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ae6899-d728-4f8e-a8f5-735787d23ee8 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:29.491500 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ae6899-d728-4f8e-a8f5-735787d23ee8 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:29.491814 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ae6899-d728-4f8e-a8f5-735787d23ee8 None admin] PATH_INFO: `/v3/users/8ac3487a22ac43de9c5668b3aa68ad44` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:29.510089 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ae6899-d728-4f8e-a8f5-735787d23ee8 None admin] RBAC: Authorizing `identity:get_user(user_id=8ac3487a22ac43de9c5668b3aa68ad44)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:29.510722 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ae6899-d728-4f8e-a8f5-735787d23ee8 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:29.512843 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 151/303] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Jul 25 12:01:29 2022] GET /identity/v3/users/8ac3487a22ac43de9c5668b3aa68ad44 => generated 293 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:01:29.518645 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1875e21e-54a1-4424-8dd0-ca49cdead8bd None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:29.628302 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1875e21e-54a1-4424-8dd0-ca49cdead8bd None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:29.664069 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8d578a-75ae-4e4c-8477-89684e993ed1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:29.664734 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8d578a-75ae-4e4c-8477-89684e993ed1 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:29.665028 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8d578a-75ae-4e4c-8477-89684e993ed1 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:29.665569 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8d578a-75ae-4e4c-8477-89684e993ed1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:29.666421 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8d578a-75ae-4e4c-8477-89684e993ed1 None admin] RBAC: Adding query filter params (role.id=716f62e98b0c4df69e1441697434334c, user.id=8ac3487a22ac43de9c5668b3aa68ad44, scope.system=all) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:01:29.666743 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8d578a-75ae-4e4c-8477-89684e993ed1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:29.668224 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8d578a-75ae-4e4c-8477-89684e993ed1 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:29.674288 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 153/304] 10.222.0.42 () {56 vars in 1308 bytes} [Mon Jul 25 12:01:29 2022] GET /identity/v3/role_assignments?user.id=8ac3487a22ac43de9c5668b3aa68ad44&scope.system=all&role.id=716f62e98b0c4df69e1441697434334c => generated 226 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:01:29.787152 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cb7adb-877a-49fc-b3cf-59b78b95b0b4 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:29.787487 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cb7adb-877a-49fc-b3cf-59b78b95b0b4 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:29.787751 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cb7adb-877a-49fc-b3cf-59b78b95b0b4 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:29.789432 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 152/305] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 25 12:01:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:01:29.795464 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9be7105-1469-46e0-bc29-062933a275fc None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:29.795824 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9be7105-1469-46e0-bc29-062933a275fc None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:29.796082 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9be7105-1469-46e0-bc29-062933a275fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:29.816769 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-a9be7105-1469-46e0-bc29-062933a275fc None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:01:29.818008 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-a9be7105-1469-46e0-bc29-062933a275fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:01:29.847222 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:29.893366 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 154/306] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 25 12:01:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 12:01:29.907424 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856bf195-9932-4313-b238-f180307e41b7 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:29.908057 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856bf195-9932-4313-b238-f180307e41b7 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:29.908057 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856bf195-9932-4313-b238-f180307e41b7 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:29.910584 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 153/307] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 25 12:01:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:01:29.914869 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53456687-c813-4bc3-952c-ec8e219573ee None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:29.915169 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53456687-c813-4bc3-952c-ec8e219573ee None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:29.915786 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53456687-c813-4bc3-952c-ec8e219573ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:29.917038 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856bf195-9932-4313-b238-f180307e41b7 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:29.934671 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-53456687-c813-4bc3-952c-ec8e219573ee None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:01:29.936227 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-53456687-c813-4bc3-952c-ec8e219573ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:01:30.027685 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 155/308] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 25 12:01:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 12:01:30.029190 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:30.037349 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856bf195-9932-4313-b238-f180307e41b7 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:30.046436 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53456687-c813-4bc3-952c-ec8e219573ee None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:30.075609 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b329a8b-7b31-47fd-a7a2-a3f54e637d9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:30.076759 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b329a8b-7b31-47fd-a7a2-a3f54e637d9e None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:30.077288 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b329a8b-7b31-47fd-a7a2-a3f54e637d9e None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:30.077786 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b329a8b-7b31-47fd-a7a2-a3f54e637d9e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:30.090942 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b329a8b-7b31-47fd-a7a2-a3f54e637d9e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 12:01:30.099344 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-2b329a8b-7b31-47fd-a7a2-a3f54e637d9e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 25 12:01:30.099344 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:01:30.099344 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:01:30.099344 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:01:30.099344 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:01:30.099344 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:01:30.099344 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:01:30.099344 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:01:30.099344 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:01:30.099344 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:01:30.099344 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:01:30.099344 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:01:30.099344 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 12:01:30.099344 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 12:01:30.099344 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 12:01:30.099344 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 12:01:30.100267 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 12:01:30.100267 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 12:01:30.100267 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 12:01:30.100267 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 12:01:30.100267 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:01:30.100267 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:01:30.100267 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:01:30.100267 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:01:30.100267 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:01:30.100267 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:01:30.100267 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:01:30.100267 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 25 12:01:30.100267 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:01:30.100267 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:01:30.100267 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:01:30.100267 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:01:30.101051 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:01:30.101051 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:01:30.101051 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:01:30.101051 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:01:30.101051 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 12:01:30.101051 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 12:01:30.101051 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 12:01:30.101051 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 12:01:30.101051 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 12:01:30.101051 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 12:01:30.101051 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 25 12:01:30.101051 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 25 12:01:30.107823 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 154/309] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Jul 25 12:01:29 2022] GET /identity/v3/roles/reader => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 12:01:30.109664 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b329a8b-7b31-47fd-a7a2-a3f54e637d9e None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:30.146899 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:30.156600 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53456687-c813-4bc3-952c-ec8e219573ee None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:30.195390 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716cecef-4daf-441a-a55e-e7e0649ac71f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:30.196320 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716cecef-4daf-441a-a55e-e7e0649ac71f None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:30.196521 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716cecef-4daf-441a-a55e-e7e0649ac71f None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:30.196795 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716cecef-4daf-441a-a55e-e7e0649ac71f None admin] PATH_INFO: `/v3/roles/member` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:30.208333 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716cecef-4daf-441a-a55e-e7e0649ac71f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 12:01:30.216637 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-716cecef-4daf-441a-a55e-e7e0649ac71f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 25 12:01:30.216637 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:01:30.216637 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:01:30.216637 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:01:30.216637 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:01:30.216637 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:01:30.216637 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:01:30.216637 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:01:30.216637 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:01:30.216637 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:01:30.216637 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:01:30.216637 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:01:30.216637 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 12:01:30.216637 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 12:01:30.216637 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 12:01:30.216637 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 12:01:30.217545 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 12:01:30.217545 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 12:01:30.217545 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 12:01:30.217545 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 12:01:30.217545 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:01:30.217545 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:01:30.217545 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:01:30.217545 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:01:30.217545 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:01:30.217545 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:01:30.217545 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:01:30.217545 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jul 25 12:01:30.217545 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:01:30.217545 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:01:30.217545 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:01:30.217545 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:01:30.218329 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:01:30.218329 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:01:30.218329 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:01:30.218329 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:01:30.218329 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 12:01:30.218329 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 12:01:30.218329 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 12:01:30.218329 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 12:01:30.218329 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 12:01:30.218329 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 12:01:30.218329 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 25 12:01:30.218329 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jul 25 12:01:30.222064 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b329a8b-7b31-47fd-a7a2-a3f54e637d9e None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:30.223271 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 156/310] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Jul 25 12:01:30 2022] GET /identity/v3/roles/member => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:01:30.224605 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716cecef-4daf-441a-a55e-e7e0649ac71f None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:30.257272 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706e4bdf-bf22-4aac-871c-c941d93e5c3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:30.258247 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706e4bdf-bf22-4aac-871c-c941d93e5c3c None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:30.258756 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706e4bdf-bf22-4aac-871c-c941d93e5c3c None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:30.259231 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706e4bdf-bf22-4aac-871c-c941d93e5c3c None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:30.260261 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706e4bdf-bf22-4aac-871c-c941d93e5c3c None admin] RBAC: Adding query filter params (name=reader) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:01:30.260768 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706e4bdf-bf22-4aac-871c-c941d93e5c3c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:30.262463 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706e4bdf-bf22-4aac-871c-c941d93e5c3c None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:30.275699 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 155/311] 10.222.0.42 () {56 vars in 1101 bytes} [Mon Jul 25 12:01:30 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 12:01:30.277818 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706e4bdf-bf22-4aac-871c-c941d93e5c3c None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:30.295200 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:30.334461 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716cecef-4daf-441a-a55e-e7e0649ac71f None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:30.369509 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c01bb1-ed7e-42c8-b682-bda0f7d4614b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:30.370446 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c01bb1-ed7e-42c8-b682-bda0f7d4614b None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:30.370922 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c01bb1-ed7e-42c8-b682-bda0f7d4614b None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:30.371371 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c01bb1-ed7e-42c8-b682-bda0f7d4614b None admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:30.372317 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c01bb1-ed7e-42c8-b682-bda0f7d4614b None admin] RBAC: Adding query filter params (name=member) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:01:30.372930 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c01bb1-ed7e-42c8-b682-bda0f7d4614b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:30.374498 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c01bb1-ed7e-42c8-b682-bda0f7d4614b None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:30.383667 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706e4bdf-bf22-4aac-871c-c941d93e5c3c None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:30.388792 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 157/312] 10.222.0.42 () {56 vars in 1101 bytes} [Mon Jul 25 12:01:30 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 12:01:30.390169 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c01bb1-ed7e-42c8-b682-bda0f7d4614b None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:30.406537 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:30.418972 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51303f98-ab04-4630-9caf-e5ce110d2206 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:30.419966 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51303f98-ab04-4630-9caf-e5ce110d2206 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:30.420374 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51303f98-ab04-4630-9caf-e5ce110d2206 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:30.420745 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51303f98-ab04-4630-9caf-e5ce110d2206 None admin] PATH_INFO: `/v3/users/07ac60e87012498b993322670ab16f61` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:30.422029 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51303f98-ab04-4630-9caf-e5ce110d2206 None admin] RBAC: Authorizing `identity:get_user(user_id=07ac60e87012498b993322670ab16f61)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:30.424159 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51303f98-ab04-4630-9caf-e5ce110d2206 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:30.430470 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 156/313] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Jul 25 12:01:30 2022] GET /identity/v3/users/07ac60e87012498b993322670ab16f61 => generated 297 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 12:01:30.431698 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51303f98-ab04-4630-9caf-e5ce110d2206 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:30.500042 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c01bb1-ed7e-42c8-b682-bda0f7d4614b None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:30.523933 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:30.535011 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51303f98-ab04-4630-9caf-e5ce110d2206 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:30.536255 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0137865c-84f3-4f15-9c77-1f7b81b6818a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:30.537396 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0137865c-84f3-4f15-9c77-1f7b81b6818a None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:30.537876 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0137865c-84f3-4f15-9c77-1f7b81b6818a None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:30.538341 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0137865c-84f3-4f15-9c77-1f7b81b6818a None admin] PATH_INFO: `/v3/users/6c8e5f6d7cb346568741bfa75792bb14` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:30.539856 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0137865c-84f3-4f15-9c77-1f7b81b6818a None admin] RBAC: Authorizing `identity:get_user(user_id=6c8e5f6d7cb346568741bfa75792bb14)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:30.541765 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0137865c-84f3-4f15-9c77-1f7b81b6818a None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:30.543887 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 158/314] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Jul 25 12:01:30 2022] GET /identity/v3/users/6c8e5f6d7cb346568741bfa75792bb14 => generated 293 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 12:01:30.550177 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0137865c-84f3-4f15-9c77-1f7b81b6818a None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:30.571360 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd42b481-b1cb-48de-85bb-cbe4211d7e10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:30.572113 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd42b481-b1cb-48de-85bb-cbe4211d7e10 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:30.572406 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd42b481-b1cb-48de-85bb-cbe4211d7e10 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:30.572873 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd42b481-b1cb-48de-85bb-cbe4211d7e10 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:30.639226 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:30.645300 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd42b481-b1cb-48de-85bb-cbe4211d7e10 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:30.646931 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd42b481-b1cb-48de-85bb-cbe4211d7e10 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:30.660240 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0137865c-84f3-4f15-9c77-1f7b81b6818a None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:30.696252 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1538cd1-469a-4faa-99cd-d92d727f16ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:30.697434 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1538cd1-469a-4faa-99cd-d92d727f16ef None admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:30.697932 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1538cd1-469a-4faa-99cd-d92d727f16ef None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:30.698406 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1538cd1-469a-4faa-99cd-d92d727f16ef None admin] PATH_INFO: `/v3/system/users/6c8e5f6d7cb346568741bfa75792bb14/roles/472313ae69174578824df0bed6390744` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:30.700160 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1538cd1-469a-4faa-99cd-d92d727f16ef None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=6c8e5f6d7cb346568741bfa75792bb14, role_id=472313ae69174578824df0bed6390744)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:30.701722 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1538cd1-469a-4faa-99cd-d92d727f16ef None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:30.713548 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 159/315] 10.222.0.42 () {58 vars in 1334 bytes} [Mon Jul 25 12:01:30 2022] PUT /identity/v3/system/users/6c8e5f6d7cb346568741bfa75792bb14/roles/472313ae69174578824df0bed6390744 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 12:01:30.736070 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:30.759653 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 157/316] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Jul 25 12:01:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 12:01:30.765434 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1538cd1-469a-4faa-99cd-d92d727f16ef None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:30.885702 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1538cd1-469a-4faa-99cd-d92d727f16ef None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:30.922794 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3556f61-6dc4-42d4-a383-468e56efe007 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:30.924543 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3556f61-6dc4-42d4-a383-468e56efe007 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:30.925023 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3556f61-6dc4-42d4-a383-468e56efe007 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:30.925513 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3556f61-6dc4-42d4-a383-468e56efe007 None admin] PATH_INFO: `/v3/projects/c980d1440d8f49049c8140fffd0d1e15` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:30.926910 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3556f61-6dc4-42d4-a383-468e56efe007 None admin] RBAC: Authorizing `identity:get_project(project_id=c980d1440d8f49049c8140fffd0d1e15)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:30.928575 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3556f61-6dc4-42d4-a383-468e56efe007 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:30.930834 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 160/317] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Jul 25 12:01:30 2022] GET /identity/v3/projects/c980d1440d8f49049c8140fffd0d1e15 => generated 302 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:01:30.938264 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd42b481-b1cb-48de-85bb-cbe4211d7e10 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:31.040606 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd42b481-b1cb-48de-85bb-cbe4211d7e10 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:31.076084 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db24e12-140c-4f03-867a-419ca94a7a79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:31.076746 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db24e12-140c-4f03-867a-419ca94a7a79 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:31.077028 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db24e12-140c-4f03-867a-419ca94a7a79 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:31.077304 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db24e12-140c-4f03-867a-419ca94a7a79 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:31.078152 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db24e12-140c-4f03-867a-419ca94a7a79 None admin] RBAC: Adding query filter params (role.id=716f62e98b0c4df69e1441697434334c, scope.project.id=c980d1440d8f49049c8140fffd0d1e15, user.id=07ac60e87012498b993322670ab16f61) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:01:31.078465 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db24e12-140c-4f03-867a-419ca94a7a79 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:31.080520 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db24e12-140c-4f03-867a-419ca94a7a79 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:31.087125 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 158/318] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Jul 25 12:01:30 2022] GET /identity/v3/role_assignments?user.id=07ac60e87012498b993322670ab16f61&scope.project.id=c980d1440d8f49049c8140fffd0d1e15&role.id=716f62e98b0c4df69e1441697434334c => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:01:31.514695 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5816e689-332c-49b0-8f16-9854b1fa4355 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:31.515093 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5816e689-332c-49b0-8f16-9854b1fa4355 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:31.515272 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5816e689-332c-49b0-8f16-9854b1fa4355 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:31.517136 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 161/319] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 25 12:01:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:01:31.522389 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b806fd-fc1d-44ba-9479-4ecdedf3a799 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:31.522686 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b806fd-fc1d-44ba-9479-4ecdedf3a799 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:31.522971 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b806fd-fc1d-44ba-9479-4ecdedf3a799 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:31.541911 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-95b806fd-fc1d-44ba-9479-4ecdedf3a799 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:01:31.543382 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-95b806fd-fc1d-44ba-9479-4ecdedf3a799 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:01:31.587502 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:31.613784 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 159/320] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 25 12:01:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 12:01:31.635472 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5816e689-332c-49b0-8f16-9854b1fa4355 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:31.652493 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:31.697513 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2378ee29-133b-4e2d-b3aa-741750f7f45a None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:31.697806 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2378ee29-133b-4e2d-b3aa-741750f7f45a None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:31.698085 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2378ee29-133b-4e2d-b3aa-741750f7f45a None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:31.699888 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 160/321] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 25 12:01:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 12:01:31.705186 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9cdcaf-072c-4ec2-ad37-c894fd17f5c1 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:31.705480 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9cdcaf-072c-4ec2-ad37-c894fd17f5c1 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:31.705762 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9cdcaf-072c-4ec2-ad37-c894fd17f5c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:31.723978 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-7a9cdcaf-072c-4ec2-ad37-c894fd17f5c1 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:01:31.725222 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-7a9cdcaf-072c-4ec2-ad37-c894fd17f5c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:01:31.744329 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5816e689-332c-49b0-8f16-9854b1fa4355 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:31.783808 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341fba03-bfe5-4cf6-a337-b69a2ea72f04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:31.784642 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341fba03-bfe5-4cf6-a337-b69a2ea72f04 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:31.784936 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341fba03-bfe5-4cf6-a337-b69a2ea72f04 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:31.785219 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341fba03-bfe5-4cf6-a337-b69a2ea72f04 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:31.786393 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341fba03-bfe5-4cf6-a337-b69a2ea72f04 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:31.787854 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341fba03-bfe5-4cf6-a337-b69a2ea72f04 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:31.789635 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 162/322] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jul 25 12:01:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:01:31.795104 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341fba03-bfe5-4cf6-a337-b69a2ea72f04 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:31.812855 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 161/323] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 25 12:01:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 25 12:01:31.834087 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9cdcaf-072c-4ec2-ad37-c894fd17f5c1 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:31.884934 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:31.903904 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341fba03-bfe5-4cf6-a337-b69a2ea72f04 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:31.941144 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041b487e-2155-49bf-b713-0fa20deabc85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:31.941824 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041b487e-2155-49bf-b713-0fa20deabc85 None admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:31.942116 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041b487e-2155-49bf-b713-0fa20deabc85 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:31.942395 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041b487e-2155-49bf-b713-0fa20deabc85 None admin] PATH_INFO: `/v3/groups` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:31.943092 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041b487e-2155-49bf-b713-0fa20deabc85 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:31.944617 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041b487e-2155-49bf-b713-0fa20deabc85 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:31.945601 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9cdcaf-072c-4ec2-ad37-c894fd17f5c1 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:31.953375 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:31.971576 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 163/324] 10.222.0.42 () {60 vars in 1134 bytes} [Mon Jul 25 12:01:31 2022] POST /identity/v3/groups => generated 226 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 12:01:31.992508 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb6e727-78d3-4700-98a5-491a983a1b23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:31.993442 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb6e727-78d3-4700-98a5-491a983a1b23 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:31.993798 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb6e727-78d3-4700-98a5-491a983a1b23 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:31.994237 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb6e727-78d3-4700-98a5-491a983a1b23 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:32.007080 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb6e727-78d3-4700-98a5-491a983a1b23 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 12:01:32.015192 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-2cb6e727-78d3-4700-98a5-491a983a1b23 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 25 12:01:32.015192 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:01:32.015192 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:01:32.015192 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:01:32.015192 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:01:32.015192 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:01:32.015192 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:01:32.015192 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:01:32.015192 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:01:32.015192 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:01:32.015192 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:01:32.015192 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:01:32.015192 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 12:01:32.015192 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 12:01:32.015192 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 12:01:32.015192 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 12:01:32.016192 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 12:01:32.016192 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 12:01:32.016192 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 12:01:32.016192 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 12:01:32.016192 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:01:32.016192 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:01:32.016192 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:01:32.016192 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:01:32.016192 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:01:32.016192 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:01:32.016192 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:01:32.016192 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 25 12:01:32.016192 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:01:32.016192 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:01:32.016192 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:01:32.016192 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:01:32.017563 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:01:32.017563 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:01:32.017563 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:01:32.017563 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:01:32.017563 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 12:01:32.017563 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 12:01:32.017563 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 12:01:32.017563 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 12:01:32.017563 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 12:01:32.017563 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 12:01:32.017563 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 25 12:01:32.017563 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 25 12:01:32.019905 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 162/325] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Jul 25 12:01:31 2022] GET /identity/v3/roles/reader => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 12:01:32.023081 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041b487e-2155-49bf-b713-0fa20deabc85 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:32.132179 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041b487e-2155-49bf-b713-0fa20deabc85 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:32.167035 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ee1d0e-2e7a-4a40-87d0-2f7cbdbd3179 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:32.167739 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ee1d0e-2e7a-4a40-87d0-2f7cbdbd3179 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:32.168078 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ee1d0e-2e7a-4a40-87d0-2f7cbdbd3179 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:32.168315 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ee1d0e-2e7a-4a40-87d0-2f7cbdbd3179 None admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:32.169069 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ee1d0e-2e7a-4a40-87d0-2f7cbdbd3179 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:01:32.169380 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ee1d0e-2e7a-4a40-87d0-2f7cbdbd3179 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:32.170760 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ee1d0e-2e7a-4a40-87d0-2f7cbdbd3179 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:32.179510 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 164/326] 10.222.0.42 () {56 vars in 1101 bytes} [Mon Jul 25 12:01:32 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 12:01:32.185004 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb6e727-78d3-4700-98a5-491a983a1b23 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:32.303154 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb6e727-78d3-4700-98a5-491a983a1b23 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:32.339980 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4bfd7e-6839-42a9-af6f-6aa64b33c6aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:32.340874 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4bfd7e-6839-42a9-af6f-6aa64b33c6aa None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:32.341158 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4bfd7e-6839-42a9-af6f-6aa64b33c6aa None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:32.341436 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4bfd7e-6839-42a9-af6f-6aa64b33c6aa None admin] PATH_INFO: `/v3/users/8ac3487a22ac43de9c5668b3aa68ad44` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:32.342663 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4bfd7e-6839-42a9-af6f-6aa64b33c6aa None admin] RBAC: Authorizing `identity:get_user(user_id=8ac3487a22ac43de9c5668b3aa68ad44)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:32.344154 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4bfd7e-6839-42a9-af6f-6aa64b33c6aa None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:32.346577 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 163/327] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Jul 25 12:01:32 2022] GET /identity/v3/users/8ac3487a22ac43de9c5668b3aa68ad44 => generated 293 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:01:32.352105 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ee1d0e-2e7a-4a40-87d0-2f7cbdbd3179 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:32.357032 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:32.483136 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ee1d0e-2e7a-4a40-87d0-2f7cbdbd3179 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:32.524709 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d420c57c-837c-41a3-bde6-5d345198ce6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:32.525955 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d420c57c-837c-41a3-bde6-5d345198ce6e None admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:32.526442 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d420c57c-837c-41a3-bde6-5d345198ce6e None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:32.526954 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d420c57c-837c-41a3-bde6-5d345198ce6e None admin] PATH_INFO: `/v3/system/users/8ac3487a22ac43de9c5668b3aa68ad44/roles/716f62e98b0c4df69e1441697434334c` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:32.528752 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d420c57c-837c-41a3-bde6-5d345198ce6e None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=8ac3487a22ac43de9c5668b3aa68ad44, role_id=716f62e98b0c4df69e1441697434334c)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:32.530373 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d420c57c-837c-41a3-bde6-5d345198ce6e None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:32.543306 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 165/328] 10.222.0.42 () {58 vars in 1334 bytes} [Mon Jul 25 12:01:32 2022] PUT /identity/v3/system/users/8ac3487a22ac43de9c5668b3aa68ad44/roles/716f62e98b0c4df69e1441697434334c => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:01:32.793396 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56989da-3eb6-47a9-bedb-b41791b5b118 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:32.793699 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56989da-3eb6-47a9-bedb-b41791b5b118 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:32.793966 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56989da-3eb6-47a9-bedb-b41791b5b118 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:32.795713 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 164/329] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 25 12:01:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:01:32.800881 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8248c5-161a-4074-8cbf-b4b84f586062 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:32.801473 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8248c5-161a-4074-8cbf-b4b84f586062 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:32.801978 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8248c5-161a-4074-8cbf-b4b84f586062 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:32.821337 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-0d8248c5-161a-4074-8cbf-b4b84f586062 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:01:32.822860 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-0d8248c5-161a-4074-8cbf-b4b84f586062 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:01:32.893965 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:32.901371 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 166/330] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 25 12:01:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 12:01:32.925282 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56989da-3eb6-47a9-bedb-b41791b5b118 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:32.975185 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:33.049406 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56989da-3eb6-47a9-bedb-b41791b5b118 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:33.084134 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54f2353-74ed-4c3d-b689-bc16fd0b2a10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:33.084945 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54f2353-74ed-4c3d-b689-bc16fd0b2a10 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:33.085230 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54f2353-74ed-4c3d-b689-bc16fd0b2a10 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:33.085509 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54f2353-74ed-4c3d-b689-bc16fd0b2a10 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:33.096169 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54f2353-74ed-4c3d-b689-bc16fd0b2a10 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 12:01:33.103783 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-c54f2353-74ed-4c3d-b689-bc16fd0b2a10 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 25 12:01:33.103783 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:01:33.103783 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:01:33.103783 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:01:33.103783 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:01:33.103783 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:01:33.103783 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:01:33.103783 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:01:33.103783 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:01:33.103783 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:01:33.103783 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:01:33.103783 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:01:33.103783 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 12:01:33.103783 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 12:01:33.103783 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 12:01:33.103783 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 12:01:33.104666 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 12:01:33.104666 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 12:01:33.104666 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 12:01:33.104666 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 12:01:33.104666 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:01:33.104666 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:01:33.104666 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:01:33.104666 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:01:33.104666 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:01:33.104666 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:01:33.104666 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:01:33.104666 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 25 12:01:33.104666 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:01:33.104666 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:01:33.104666 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:01:33.104666 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:01:33.105454 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:01:33.105454 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:01:33.105454 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:01:33.105454 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:01:33.105454 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 12:01:33.105454 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 12:01:33.105454 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 12:01:33.105454 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 12:01:33.105454 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 12:01:33.105454 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 12:01:33.105454 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 25 12:01:33.105454 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 25 12:01:33.105454 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 165/331] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Jul 25 12:01:32 2022] GET /identity/v3/roles/member => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:01:33.112624 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8248c5-161a-4074-8cbf-b4b84f586062 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:33.241884 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8248c5-161a-4074-8cbf-b4b84f586062 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:33.284179 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cd29ce-509b-40fa-9067-f60ba5b254d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:33.284869 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cd29ce-509b-40fa-9067-f60ba5b254d5 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:33.285163 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cd29ce-509b-40fa-9067-f60ba5b254d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:33.285451 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cd29ce-509b-40fa-9067-f60ba5b254d5 None admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:33.286219 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80cd29ce-509b-40fa-9067-f60ba5b254d5 None admin] RBAC: Adding query filter params (name=member) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:01:33.286537 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80cd29ce-509b-40fa-9067-f60ba5b254d5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:33.287940 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80cd29ce-509b-40fa-9067-f60ba5b254d5 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:33.303605 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 167/332] 10.222.0.42 () {56 vars in 1101 bytes} [Mon Jul 25 12:01:33 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:01:33.309155 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54f2353-74ed-4c3d-b689-bc16fd0b2a10 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:33.435920 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:33.442045 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54f2353-74ed-4c3d-b689-bc16fd0b2a10 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:33.481141 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c9d252-488d-474c-9b12-6703ba8cab73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:33.482092 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c9d252-488d-474c-9b12-6703ba8cab73 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:33.482379 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c9d252-488d-474c-9b12-6703ba8cab73 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:33.482655 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c9d252-488d-474c-9b12-6703ba8cab73 None admin] PATH_INFO: `/v3/users/6c8e5f6d7cb346568741bfa75792bb14` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:33.483924 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c9d252-488d-474c-9b12-6703ba8cab73 None admin] RBAC: Authorizing `identity:get_user(user_id=6c8e5f6d7cb346568741bfa75792bb14)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:33.485405 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c9d252-488d-474c-9b12-6703ba8cab73 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:33.487454 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 166/333] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Jul 25 12:01:33 2022] GET /identity/v3/users/6c8e5f6d7cb346568741bfa75792bb14 => generated 293 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:01:33.493770 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cd29ce-509b-40fa-9067-f60ba5b254d5 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:33.575626 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:33.626109 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cd29ce-509b-40fa-9067-f60ba5b254d5 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:33.663013 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1febfe-8150-4b49-92b2-6e5dec0569de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:33.663837 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1febfe-8150-4b49-92b2-6e5dec0569de None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:33.664131 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1febfe-8150-4b49-92b2-6e5dec0569de None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:33.664414 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1febfe-8150-4b49-92b2-6e5dec0569de None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:33.665262 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1febfe-8150-4b49-92b2-6e5dec0569de None admin] RBAC: Adding query filter params (role.id=472313ae69174578824df0bed6390744, user.id=6c8e5f6d7cb346568741bfa75792bb14, scope.system=all) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:01:33.665975 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1febfe-8150-4b49-92b2-6e5dec0569de None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:33.668129 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1febfe-8150-4b49-92b2-6e5dec0569de None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:33.683000 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 168/334] 10.222.0.42 () {56 vars in 1308 bytes} [Mon Jul 25 12:01:33 2022] GET /identity/v3/role_assignments?user.id=6c8e5f6d7cb346568741bfa75792bb14&scope.system=all&role.id=472313ae69174578824df0bed6390744 => generated 510 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:01:34.616678 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084f5ba3-45e7-4187-a831-a75bdbac9749 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:34.617182 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084f5ba3-45e7-4187-a831-a75bdbac9749 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:34.617255 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084f5ba3-45e7-4187-a831-a75bdbac9749 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:34.618869 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 167/335] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 25 12:01:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:01:34.623872 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbeb48f6-f715-4666-afaf-9c2cea60651f None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:34.624268 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbeb48f6-f715-4666-afaf-9c2cea60651f None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:34.624432 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbeb48f6-f715-4666-afaf-9c2cea60651f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:34.648555 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-bbeb48f6-f715-4666-afaf-9c2cea60651f None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:01:34.649763 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-bbeb48f6-f715-4666-afaf-9c2cea60651f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:01:34.730887 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 169/336] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 25 12:01:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 12:01:34.757534 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76867bd4-1f03-4916-bb2b-ef320349b4d7 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:34.757827 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76867bd4-1f03-4916-bb2b-ef320349b4d7 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:34.758109 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76867bd4-1f03-4916-bb2b-ef320349b4d7 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:34.759473 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 168/337] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 25 12:01:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:01:34.764738 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b9f229-a391-43d5-936c-18af6bcea5ad None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:34.765028 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b9f229-a391-43d5-936c-18af6bcea5ad None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:34.765309 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b9f229-a391-43d5-936c-18af6bcea5ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:34.780800 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76867bd4-1f03-4916-bb2b-ef320349b4d7 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:34.808236 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-d1b9f229-a391-43d5-936c-18af6bcea5ad None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:01:34.815696 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-d1b9f229-a391-43d5-936c-18af6bcea5ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:01:34.912024 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:34.952968 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 170/338] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 25 12:01:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 12:01:34.992630 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b9f229-a391-43d5-936c-18af6bcea5ad None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:35.024336 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76867bd4-1f03-4916-bb2b-ef320349b4d7 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:35.084368 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5163b49-2cc8-4c41-b5fb-26b95fde7668 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:35.085187 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5163b49-2cc8-4c41-b5fb-26b95fde7668 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:35.085476 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5163b49-2cc8-4c41-b5fb-26b95fde7668 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:35.085756 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5163b49-2cc8-4c41-b5fb-26b95fde7668 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:35.116549 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5163b49-2cc8-4c41-b5fb-26b95fde7668 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 12:01:35.117334 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b9f229-a391-43d5-936c-18af6bcea5ad None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:35.124503 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-a5163b49-2cc8-4c41-b5fb-26b95fde7668 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 25 12:01:35.124503 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:01:35.124503 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:01:35.124503 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:01:35.124503 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:01:35.124503 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:01:35.124503 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:01:35.124503 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:01:35.124503 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:01:35.124503 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:01:35.124503 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:01:35.124503 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:01:35.124503 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 12:01:35.124503 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 12:01:35.124503 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 12:01:35.124503 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 12:01:35.125395 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 12:01:35.125395 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 12:01:35.125395 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 12:01:35.125395 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 12:01:35.125395 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:01:35.125395 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:01:35.125395 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:01:35.125395 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:01:35.125395 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:01:35.125395 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:01:35.125395 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:01:35.125395 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 25 12:01:35.125395 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:01:35.125395 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:01:35.125395 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:01:35.125395 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:01:35.127165 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:01:35.127165 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:01:35.127165 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:01:35.127165 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:01:35.127165 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 12:01:35.127165 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 12:01:35.127165 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 12:01:35.127165 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 12:01:35.127165 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 12:01:35.127165 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 12:01:35.127165 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 25 12:01:35.127165 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 25 12:01:35.127165 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 169/339] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Jul 25 12:01:34 2022] GET /identity/v3/roles/member => generated 84 bytes in 346 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 12:01:35.134377 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5163b49-2cc8-4c41-b5fb-26b95fde7668 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:35.168580 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067d02ee-7d74-4342-a65a-40b2bc6c0da3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:35.169751 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067d02ee-7d74-4342-a65a-40b2bc6c0da3 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:35.170243 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067d02ee-7d74-4342-a65a-40b2bc6c0da3 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:35.170738 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067d02ee-7d74-4342-a65a-40b2bc6c0da3 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:35.189319 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067d02ee-7d74-4342-a65a-40b2bc6c0da3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 12:01:35.199025 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-067d02ee-7d74-4342-a65a-40b2bc6c0da3 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 12:01:35.199025 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:01:35.199025 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:01:35.199025 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:01:35.199025 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:01:35.199025 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:01:35.199025 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:01:35.199025 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:01:35.199025 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:01:35.199025 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:01:35.199025 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:01:35.199025 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:01:35.199025 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 12:01:35.199025 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 12:01:35.199025 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 12:01:35.199025 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 12:01:35.199950 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 12:01:35.199950 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 12:01:35.199950 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 12:01:35.199950 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 12:01:35.199950 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:01:35.199950 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:01:35.199950 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:01:35.199950 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:01:35.199950 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:01:35.199950 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:01:35.199950 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:01:35.199950 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jul 25 12:01:35.199950 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:01:35.199950 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:01:35.199950 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:01:35.199950 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:01:35.200753 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:01:35.200753 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:01:35.200753 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:01:35.200753 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:01:35.200753 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 12:01:35.200753 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 12:01:35.200753 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 12:01:35.200753 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 12:01:35.200753 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 12:01:35.200753 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 12:01:35.200753 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 12:01:35.200753 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jul 25 12:01:35.203496 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 171/340] 10.222.0.42 () {56 vars in 1114 bytes} [Mon Jul 25 12:01:34 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:01:35.207922 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:35.221937 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067d02ee-7d74-4342-a65a-40b2bc6c0da3 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:35.299375 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5163b49-2cc8-4c41-b5fb-26b95fde7668 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:35.299908 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:35.336158 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976d8974-4605-46cb-9953-b21655915e83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:35.336840 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976d8974-4605-46cb-9953-b21655915e83 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:35.337133 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976d8974-4605-46cb-9953-b21655915e83 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:35.337418 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976d8974-4605-46cb-9953-b21655915e83 None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:35.338179 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976d8974-4605-46cb-9953-b21655915e83 None admin] RBAC: Adding query filter params (name=member) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:01:35.338490 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976d8974-4605-46cb-9953-b21655915e83 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:35.339919 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976d8974-4605-46cb-9953-b21655915e83 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:35.348984 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 170/341] 10.222.0.42 () {56 vars in 1101 bytes} [Mon Jul 25 12:01:35 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:01:35.361276 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976d8974-4605-46cb-9953-b21655915e83 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:35.419439 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067d02ee-7d74-4342-a65a-40b2bc6c0da3 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:35.453737 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:35.488381 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edec6778-7453-4127-9db4-c5df055d52ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:35.489068 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edec6778-7453-4127-9db4-c5df055d52ed None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:35.489353 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edec6778-7453-4127-9db4-c5df055d52ed None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:35.489633 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edec6778-7453-4127-9db4-c5df055d52ed None admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:35.490396 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edec6778-7453-4127-9db4-c5df055d52ed None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:01:35.490710 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edec6778-7453-4127-9db4-c5df055d52ed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:35.492121 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edec6778-7453-4127-9db4-c5df055d52ed None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:35.495279 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976d8974-4605-46cb-9953-b21655915e83 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:35.517942 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 172/342] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Jul 25 12:01:35 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:01:35.525162 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edec6778-7453-4127-9db4-c5df055d52ed None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:35.576769 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802df939-e7e6-4c88-b3ba-48223484ed69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:35.577440 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802df939-e7e6-4c88-b3ba-48223484ed69 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:35.577729 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802df939-e7e6-4c88-b3ba-48223484ed69 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:35.578014 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802df939-e7e6-4c88-b3ba-48223484ed69 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:35.723845 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802df939-e7e6-4c88-b3ba-48223484ed69 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:35.726133 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802df939-e7e6-4c88-b3ba-48223484ed69 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:35.763843 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:35.767462 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edec6778-7453-4127-9db4-c5df055d52ed None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:35.848008 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eac32c7-b6d9-4f9d-a73a-111a2200665d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:35.848681 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eac32c7-b6d9-4f9d-a73a-111a2200665d None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:35.848974 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eac32c7-b6d9-4f9d-a73a-111a2200665d None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:35.849260 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eac32c7-b6d9-4f9d-a73a-111a2200665d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:35.881022 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:35.884326 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 171/343] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Jul 25 12:01:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 524 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 12:01:35.888213 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74341498-cee0-414e-bc96-fd3349baf317 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:35.888443 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74341498-cee0-414e-bc96-fd3349baf317 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:35.888820 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74341498-cee0-414e-bc96-fd3349baf317 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:35.890501 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 172/344] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 25 12:01:35 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:01:35.891841 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998cb3dd-90a8-43b0-8d1b-1cfead7566df None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:35.892131 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998cb3dd-90a8-43b0-8d1b-1cfead7566df None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:35.892408 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998cb3dd-90a8-43b0-8d1b-1cfead7566df None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:35.894503 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 173/345] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 25 12:01:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:01:35.899196 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998cb3dd-90a8-43b0-8d1b-1cfead7566df None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:35.933218 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eac32c7-b6d9-4f9d-a73a-111a2200665d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:35.934678 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eac32c7-b6d9-4f9d-a73a-111a2200665d None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:36.020076 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998cb3dd-90a8-43b0-8d1b-1cfead7566df None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:36.036929 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:36.041324 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 173/346] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Jul 25 12:01:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 516 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 12:01:36.042682 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c8df9f-5c3e-48a1-a8a5-8c6ec0d02462 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:36.042982 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c8df9f-5c3e-48a1-a8a5-8c6ec0d02462 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:36.043374 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c8df9f-5c3e-48a1-a8a5-8c6ec0d02462 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:36.066690 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-95c8df9f-5c3e-48a1-a8a5-8c6ec0d02462 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:01:36.067671 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90d1448-df8b-4e38-85fa-ec18eb521d5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:36.068018 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-95c8df9f-5c3e-48a1-a8a5-8c6ec0d02462 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:01:36.068705 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90d1448-df8b-4e38-85fa-ec18eb521d5b None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:36.069058 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90d1448-df8b-4e38-85fa-ec18eb521d5b None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:36.069405 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90d1448-df8b-4e38-85fa-ec18eb521d5b None admin] PATH_INFO: `/v3/projects/23c9d26ac78646c79204b87bd5332e2c` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:36.070674 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90d1448-df8b-4e38-85fa-ec18eb521d5b None admin] RBAC: Authorizing `identity:get_project(project_id=23c9d26ac78646c79204b87bd5332e2c)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:36.072252 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90d1448-df8b-4e38-85fa-ec18eb521d5b None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:36.074235 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 174/347] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Jul 25 12:01:35 2022] GET /identity/v3/projects/23c9d26ac78646c79204b87bd5332e2c => generated 298 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:01:36.075321 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b9c49a-c8de-481e-a85a-8bc7bfb021a0 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:36.075666 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b9c49a-c8de-481e-a85a-8bc7bfb021a0 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:36.076016 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b9c49a-c8de-481e-a85a-8bc7bfb021a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:36.098448 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-c1b9c49a-c8de-481e-a85a-8bc7bfb021a0 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:01:36.099787 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-c1b9c49a-c8de-481e-a85a-8bc7bfb021a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:01:36.151780 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 174/348] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 25 12:01:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 12:01:36.153499 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c8df9f-5c3e-48a1-a8a5-8c6ec0d02462 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:36.156279 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:36.188797 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 175/349] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 25 12:01:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 12:01:36.189724 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b9c49a-c8de-481e-a85a-8bc7bfb021a0 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:36.284307 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c8df9f-5c3e-48a1-a8a5-8c6ec0d02462 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:36.315070 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b9c49a-c8de-481e-a85a-8bc7bfb021a0 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:36.323951 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:36.331784 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084f7558-36dc-4070-b712-790e75967823 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:36.332672 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084f7558-36dc-4070-b712-790e75967823 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:36.332966 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084f7558-36dc-4070-b712-790e75967823 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:36.333526 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084f7558-36dc-4070-b712-790e75967823 None admin] PATH_INFO: `/v3/projects/23c9d26ac78646c79204b87bd5332e2c` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:36.335834 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084f7558-36dc-4070-b712-790e75967823 None admin] RBAC: Authorizing `identity:get_project(project_id=23c9d26ac78646c79204b87bd5332e2c)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:36.337448 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084f7558-36dc-4070-b712-790e75967823 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:36.340203 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 175/350] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Jul 25 12:01:36 2022] GET /identity/v3/projects/23c9d26ac78646c79204b87bd5332e2c => generated 298 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:01:36.340991 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084f7558-36dc-4070-b712-790e75967823 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:36.361902 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2b1188-3b52-43fa-a877-ae96b0b600c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:36.362927 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2b1188-3b52-43fa-a877-ae96b0b600c6 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:36.363351 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2b1188-3b52-43fa-a877-ae96b0b600c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:36.363631 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2b1188-3b52-43fa-a877-ae96b0b600c6 None admin] PATH_INFO: `/v3/groups/be589db33cdd45c3b8f0684e8ac54c2b` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:36.371578 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2b1188-3b52-43fa-a877-ae96b0b600c6 None admin] RBAC: Authorizing `identity:get_group(group_id=be589db33cdd45c3b8f0684e8ac54c2b)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:36.373289 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2b1188-3b52-43fa-a877-ae96b0b600c6 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:36.375184 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 176/351] 10.222.0.42 () {56 vars in 1180 bytes} [Mon Jul 25 12:01:36 2022] GET /identity/v3/groups/be589db33cdd45c3b8f0684e8ac54c2b => generated 226 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:01:36.376777 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2b1188-3b52-43fa-a877-ae96b0b600c6 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:36.439784 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:36.468669 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084f7558-36dc-4070-b712-790e75967823 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:36.502582 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2b1188-3b52-43fa-a877-ae96b0b600c6 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:36.507637 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa674cf-6442-4e1b-8b25-f9663cc30609 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:36.509752 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa674cf-6442-4e1b-8b25-f9663cc30609 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:36.510206 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa674cf-6442-4e1b-8b25-f9663cc30609 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:36.510654 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa674cf-6442-4e1b-8b25-f9663cc30609 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:36.523793 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa674cf-6442-4e1b-8b25-f9663cc30609 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 12:01:36.531900 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-daa674cf-6442-4e1b-8b25-f9663cc30609 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 25 12:01:36.531900 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:01:36.531900 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:01:36.531900 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:01:36.531900 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:01:36.531900 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:01:36.531900 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:01:36.531900 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:01:36.531900 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:01:36.531900 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:01:36.531900 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:01:36.531900 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:01:36.531900 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 12:01:36.531900 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 12:01:36.531900 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 12:01:36.531900 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 12:01:36.532895 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 12:01:36.532895 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 12:01:36.532895 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 12:01:36.532895 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 12:01:36.532895 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:01:36.532895 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:01:36.532895 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:01:36.532895 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:01:36.532895 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:01:36.532895 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:01:36.532895 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:01:36.532895 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jul 25 12:01:36.532895 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:01:36.532895 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:01:36.532895 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:01:36.532895 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:01:36.533712 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:01:36.533712 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:01:36.533712 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:01:36.533712 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:01:36.533712 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 12:01:36.533712 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 12:01:36.533712 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 12:01:36.533712 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 12:01:36.533712 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 12:01:36.533712 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 12:01:36.533712 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 25 12:01:36.533712 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jul 25 12:01:36.536275 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 176/352] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Jul 25 12:01:36 2022] GET /identity/v3/roles/reader => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:01:36.537286 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa674cf-6442-4e1b-8b25-f9663cc30609 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:36.545957 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d269fcd-865e-4bc1-aa5a-0f82099df978 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:36.546769 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d269fcd-865e-4bc1-aa5a-0f82099df978 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:36.547057 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d269fcd-865e-4bc1-aa5a-0f82099df978 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:36.547337 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d269fcd-865e-4bc1-aa5a-0f82099df978 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:36.559909 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d269fcd-865e-4bc1-aa5a-0f82099df978 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 12:01:36.569139 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-9d269fcd-865e-4bc1-aa5a-0f82099df978 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 25 12:01:36.569139 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:01:36.569139 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:01:36.569139 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:01:36.569139 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:01:36.569139 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:01:36.569139 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:01:36.569139 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:01:36.569139 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:01:36.569139 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:01:36.569139 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:01:36.569139 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:01:36.569139 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 12:01:36.569139 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 12:01:36.569139 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 12:01:36.569139 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 12:01:36.570836 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 12:01:36.570836 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 12:01:36.570836 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 12:01:36.570836 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 12:01:36.570836 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:01:36.570836 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:01:36.570836 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:01:36.570836 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:01:36.570836 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:01:36.570836 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:01:36.570836 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:01:36.570836 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 25 12:01:36.570836 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:01:36.570836 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:01:36.570836 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:01:36.570836 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:01:36.572367 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:01:36.572367 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:01:36.572367 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:01:36.572367 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:01:36.572367 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 12:01:36.572367 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 12:01:36.572367 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 12:01:36.572367 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 12:01:36.572367 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 12:01:36.572367 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 12:01:36.572367 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 25 12:01:36.572367 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 25 12:01:36.575559 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 177/353] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Jul 25 12:01:36 2022] GET /identity/v3/roles/member => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:01:36.576927 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d269fcd-865e-4bc1-aa5a-0f82099df978 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:36.610441 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:36.657083 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa674cf-6442-4e1b-8b25-f9663cc30609 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:36.700878 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d269fcd-865e-4bc1-aa5a-0f82099df978 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:36.708738 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509edeee-fdd4-4b3f-b572-612ff062d076 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:36.709604 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:36.710887 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509edeee-fdd4-4b3f-b572-612ff062d076 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:36.711427 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509edeee-fdd4-4b3f-b572-612ff062d076 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:36.711827 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509edeee-fdd4-4b3f-b572-612ff062d076 None admin] PATH_INFO: `/v3/groups/be589db33cdd45c3b8f0684e8ac54c2b` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:36.713778 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509edeee-fdd4-4b3f-b572-612ff062d076 None admin] RBAC: Authorizing `identity:get_group(group_id=be589db33cdd45c3b8f0684e8ac54c2b)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:36.715594 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509edeee-fdd4-4b3f-b572-612ff062d076 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:36.717089 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 177/354] 10.222.0.42 () {56 vars in 1180 bytes} [Mon Jul 25 12:01:36 2022] GET /identity/v3/groups/be589db33cdd45c3b8f0684e8ac54c2b => generated 226 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:01:36.717700 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509edeee-fdd4-4b3f-b572-612ff062d076 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:36.746518 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eba0243-c16c-4aa6-b297-c79e77c1e5d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:36.747189 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eba0243-c16c-4aa6-b297-c79e77c1e5d2 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:36.747468 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eba0243-c16c-4aa6-b297-c79e77c1e5d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:36.747793 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eba0243-c16c-4aa6-b297-c79e77c1e5d2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:36.748726 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eba0243-c16c-4aa6-b297-c79e77c1e5d2 None admin] RBAC: Adding query filter params (group.id=be589db33cdd45c3b8f0684e8ac54c2b, role.id=472313ae69174578824df0bed6390744, scope.project.id=23c9d26ac78646c79204b87bd5332e2c) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:01:36.748952 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eba0243-c16c-4aa6-b297-c79e77c1e5d2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:36.750551 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eba0243-c16c-4aa6-b297-c79e77c1e5d2 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:36.757893 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 178/355] 10.222.0.42 () {56 vars in 1376 bytes} [Mon Jul 25 12:01:36 2022] GET /identity/v3/role_assignments?group.id=be589db33cdd45c3b8f0684e8ac54c2b&scope.project.id=23c9d26ac78646c79204b87bd5332e2c&role.id=472313ae69174578824df0bed6390744 => generated 260 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:01:36.758684 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eba0243-c16c-4aa6-b297-c79e77c1e5d2 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:36.851266 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509edeee-fdd4-4b3f-b572-612ff062d076 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:36.891665 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:36.906786 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eba0243-c16c-4aa6-b297-c79e77c1e5d2 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:36.930765 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6860586-a28c-4010-a4f3-a32c499ce408 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:36.930765 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6860586-a28c-4010-a4f3-a32c499ce408 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:36.930765 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6860586-a28c-4010-a4f3-a32c499ce408 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:36.930765 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6860586-a28c-4010-a4f3-a32c499ce408 None admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:36.931185 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6860586-a28c-4010-a4f3-a32c499ce408 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:01:36.931185 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6860586-a28c-4010-a4f3-a32c499ce408 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:36.932660 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6860586-a28c-4010-a4f3-a32c499ce408 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:36.944952 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 178/356] 10.222.0.42 () {56 vars in 1101 bytes} [Mon Jul 25 12:01:36 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:01:36.946693 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19558925-be01-4a1e-8f4a-e747ccdab613 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:36.946693 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19558925-be01-4a1e-8f4a-e747ccdab613 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:36.951705 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19558925-be01-4a1e-8f4a-e747ccdab613 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:36.953197 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 179/357] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 25 12:01:36 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:01:36.954117 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19558925-be01-4a1e-8f4a-e747ccdab613 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:36.955345 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b1e621-f2fd-4ac0-a9bc-3eb177168938 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:36.956089 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b1e621-f2fd-4ac0-a9bc-3eb177168938 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:36.956380 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b1e621-f2fd-4ac0-a9bc-3eb177168938 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:36.956659 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b1e621-f2fd-4ac0-a9bc-3eb177168938 None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:36.957484 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b1e621-f2fd-4ac0-a9bc-3eb177168938 None admin] RBAC: Adding query filter params (name=member) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:01:36.957774 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b1e621-f2fd-4ac0-a9bc-3eb177168938 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:36.959171 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b1e621-f2fd-4ac0-a9bc-3eb177168938 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:36.975822 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 179/358] 10.222.0.42 () {56 vars in 1101 bytes} [Mon Jul 25 12:01:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:01:36.979676 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aab1f7e-7bae-4b2e-9d61-7887b9786110 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:36.980364 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aab1f7e-7bae-4b2e-9d61-7887b9786110 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:36.980885 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aab1f7e-7bae-4b2e-9d61-7887b9786110 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:36.983524 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 180/359] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 25 12:01:36 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:01:36.986022 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aab1f7e-7bae-4b2e-9d61-7887b9786110 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:37.023288 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:37.086360 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19558925-be01-4a1e-8f4a-e747ccdab613 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:37.123153 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aab1f7e-7bae-4b2e-9d61-7887b9786110 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:37.133726 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8d5dba-3f7d-44c2-9cfa-dd8b5629ca61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:37.134545 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8d5dba-3f7d-44c2-9cfa-dd8b5629ca61 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:37.134673 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8d5dba-3f7d-44c2-9cfa-dd8b5629ca61 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:37.134939 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8d5dba-3f7d-44c2-9cfa-dd8b5629ca61 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:37.135804 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8d5dba-3f7d-44c2-9cfa-dd8b5629ca61 None admin] RBAC: Adding query filter params (group.id=be589db33cdd45c3b8f0684e8ac54c2b, role.id=dea3ef72822043f3baf3bcc1be7c318f, scope.project.id=23c9d26ac78646c79204b87bd5332e2c) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:01:37.136118 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8d5dba-3f7d-44c2-9cfa-dd8b5629ca61 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:37.137558 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8d5dba-3f7d-44c2-9cfa-dd8b5629ca61 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:37.146491 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 180/360] 10.222.0.42 () {56 vars in 1376 bytes} [Mon Jul 25 12:01:36 2022] GET /identity/v3/role_assignments?group.id=be589db33cdd45c3b8f0684e8ac54c2b&scope.project.id=23c9d26ac78646c79204b87bd5332e2c&role.id=dea3ef72822043f3baf3bcc1be7c318f => generated 260 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:01:37.147782 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2c79ad-4a25-410e-9ec0-99981e8fbbf2 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:37.148069 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2c79ad-4a25-410e-9ec0-99981e8fbbf2 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:37.148352 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2c79ad-4a25-410e-9ec0-99981e8fbbf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:37.171174 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-2f2c79ad-4a25-410e-9ec0-99981e8fbbf2 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:01:37.172495 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-2f2c79ad-4a25-410e-9ec0-99981e8fbbf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:01:37.189110 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3451d463-bd78-41a7-a880-7fa50b26fd5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:37.190233 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3451d463-bd78-41a7-a880-7fa50b26fd5f None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:37.190528 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3451d463-bd78-41a7-a880-7fa50b26fd5f None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:37.190862 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3451d463-bd78-41a7-a880-7fa50b26fd5f None admin] PATH_INFO: `/v3/users/8ac3487a22ac43de9c5668b3aa68ad44` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:37.192293 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3451d463-bd78-41a7-a880-7fa50b26fd5f None admin] RBAC: Authorizing `identity:get_user(user_id=8ac3487a22ac43de9c5668b3aa68ad44)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:37.194089 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3451d463-bd78-41a7-a880-7fa50b26fd5f None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:37.197191 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 181/361] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Jul 25 12:01:36 2022] GET /identity/v3/users/8ac3487a22ac43de9c5668b3aa68ad44 => generated 293 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:01:37.198187 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3451d463-bd78-41a7-a880-7fa50b26fd5f None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:37.291945 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:37.292178 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 181/362] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 25 12:01:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 12:01:37.294012 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aaca7be-aaf4-426f-903d-33c30c815e26 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:37.294414 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aaca7be-aaf4-426f-903d-33c30c815e26 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:37.294772 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aaca7be-aaf4-426f-903d-33c30c815e26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:37.317781 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-1aaca7be-aaf4-426f-903d-33c30c815e26 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:01:37.319142 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-1aaca7be-aaf4-426f-903d-33c30c815e26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:01:37.351904 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:37.358730 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3451d463-bd78-41a7-a880-7fa50b26fd5f None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:37.407660 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e2aea1-3a73-42c4-bc4a-73a23554ef00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:37.408255 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e2aea1-3a73-42c4-bc4a-73a23554ef00 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:37.408600 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e2aea1-3a73-42c4-bc4a-73a23554ef00 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:37.408768 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 182/363] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 25 12:01:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 12:01:37.408920 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e2aea1-3a73-42c4-bc4a-73a23554ef00 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:37.409605 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aaca7be-aaf4-426f-903d-33c30c815e26 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:37.490160 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e2aea1-3a73-42c4-bc4a-73a23554ef00 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:37.491649 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e2aea1-3a73-42c4-bc4a-73a23554ef00 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:37.544372 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aaca7be-aaf4-426f-903d-33c30c815e26 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:37.560129 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:37.586277 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68ed79d-c753-4d01-82d8-43b650f018fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:37.586949 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68ed79d-c753-4d01-82d8-43b650f018fa None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:37.587241 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68ed79d-c753-4d01-82d8-43b650f018fa None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:37.587529 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68ed79d-c753-4d01-82d8-43b650f018fa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:37.588405 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68ed79d-c753-4d01-82d8-43b650f018fa None admin] RBAC: Adding query filter params (role.id=716f62e98b0c4df69e1441697434334c, user.id=8ac3487a22ac43de9c5668b3aa68ad44, scope.system=all) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:01:37.588727 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68ed79d-c753-4d01-82d8-43b650f018fa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:37.590176 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68ed79d-c753-4d01-82d8-43b650f018fa None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:37.597300 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:37.601471 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 182/364] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Jul 25 12:01:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 12:01:37.602272 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e2aea1-3a73-42c4-bc4a-73a23554ef00 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:37.608755 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 183/365] 10.222.0.42 () {56 vars in 1308 bytes} [Mon Jul 25 12:01:37 2022] GET /identity/v3/role_assignments?user.id=8ac3487a22ac43de9c5668b3aa68ad44&scope.system=all&role.id=716f62e98b0c4df69e1441697434334c => generated 510 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:01:37.609628 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68ed79d-c753-4d01-82d8-43b650f018fa None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:37.712832 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e2aea1-3a73-42c4-bc4a-73a23554ef00 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:37.725508 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68ed79d-c753-4d01-82d8-43b650f018fa None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:37.749940 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2543f32-de42-4d74-a746-ce9e12df44d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:37.750755 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2543f32-de42-4d74-a746-ce9e12df44d6 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:37.751043 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2543f32-de42-4d74-a746-ce9e12df44d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:37.751359 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2543f32-de42-4d74-a746-ce9e12df44d6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:37.762927 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2543f32-de42-4d74-a746-ce9e12df44d6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 12:01:37.763362 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2c52e2-b13e-4f5a-98f9-916b0b3d0cea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:37.764191 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2c52e2-b13e-4f5a-98f9-916b0b3d0cea None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:37.764480 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2c52e2-b13e-4f5a-98f9-916b0b3d0cea None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:37.764762 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2c52e2-b13e-4f5a-98f9-916b0b3d0cea None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:37.772654 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-f2543f32-de42-4d74-a746-ce9e12df44d6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 12:01:37.772654 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:01:37.772654 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:01:37.772654 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:01:37.772654 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:01:37.772654 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:01:37.772654 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:01:37.772654 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:01:37.772654 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:01:37.772654 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:01:37.772654 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:01:37.772654 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:01:37.772654 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 12:01:37.772654 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 12:01:37.772654 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 12:01:37.772654 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 12:01:37.774772 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 12:01:37.774772 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 12:01:37.774772 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 12:01:37.774772 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 12:01:37.774772 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:01:37.774772 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:01:37.774772 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:01:37.774772 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:01:37.774772 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:01:37.774772 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:01:37.774772 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:01:37.774772 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 25 12:01:37.774772 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:01:37.774772 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:01:37.774772 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:01:37.774772 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:01:37.776753 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:01:37.776753 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:01:37.776753 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:01:37.776753 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:01:37.776753 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 12:01:37.776753 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 12:01:37.776753 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 12:01:37.776753 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 12:01:37.776753 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 12:01:37.776753 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 12:01:37.776753 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 12:01:37.776753 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 25 12:01:37.776753 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 183/366] 10.222.0.42 () {56 vars in 1114 bytes} [Mon Jul 25 12:01:37 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:01:37.776753 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2543f32-de42-4d74-a746-ce9e12df44d6 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:37.778466 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2c52e2-b13e-4f5a-98f9-916b0b3d0cea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 12:01:37.787858 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-eb2c52e2-b13e-4f5a-98f9-916b0b3d0cea None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 12:01:37.787858 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:01:37.787858 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:01:37.787858 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:01:37.787858 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:01:37.787858 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:01:37.787858 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:01:37.787858 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:01:37.787858 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:01:37.787858 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:01:37.787858 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:01:37.787858 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:01:37.787858 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 12:01:37.787858 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 12:01:37.787858 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 12:01:37.787858 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 12:01:37.789890 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 12:01:37.789890 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 12:01:37.789890 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 12:01:37.789890 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 12:01:37.789890 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:01:37.789890 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:01:37.789890 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:01:37.789890 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:01:37.789890 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:01:37.789890 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:01:37.789890 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:01:37.789890 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jul 25 12:01:37.789890 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:01:37.789890 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:01:37.789890 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:01:37.789890 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:01:37.792109 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:01:37.792109 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:01:37.792109 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:01:37.792109 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:01:37.792109 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 12:01:37.792109 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 12:01:37.792109 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 12:01:37.792109 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 12:01:37.792109 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 12:01:37.792109 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 12:01:37.792109 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 12:01:37.792109 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jul 25 12:01:37.792109 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 184/367] 10.222.0.42 () {56 vars in 1096 bytes} [Mon Jul 25 12:01:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:01:37.792109 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2c52e2-b13e-4f5a-98f9-916b0b3d0cea None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:37.828399 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:37.853821 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:37.907221 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2c52e2-b13e-4f5a-98f9-916b0b3d0cea None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:37.913059 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2543f32-de42-4d74-a746-ce9e12df44d6 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:37.945937 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab4a1cc-0616-46f5-a628-ffd53b11d947 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:37.946866 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab4a1cc-0616-46f5-a628-ffd53b11d947 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:37.947303 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab4a1cc-0616-46f5-a628-ffd53b11d947 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:37.947754 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab4a1cc-0616-46f5-a628-ffd53b11d947 None admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:37.948664 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab4a1cc-0616-46f5-a628-ffd53b11d947 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:01:37.949151 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab4a1cc-0616-46f5-a628-ffd53b11d947 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:37.950700 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab4a1cc-0616-46f5-a628-ffd53b11d947 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:37.960188 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 185/368] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Jul 25 12:01:37 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:01:37.961299 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab4a1cc-0616-46f5-a628-ffd53b11d947 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:37.975414 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908b5961-82a4-452a-83a4-4753ebad0cd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:37.976234 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908b5961-82a4-452a-83a4-4753ebad0cd5 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:37.976697 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908b5961-82a4-452a-83a4-4753ebad0cd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:37.976811 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908b5961-82a4-452a-83a4-4753ebad0cd5 None admin] PATH_INFO: `/v3/projects/c980d1440d8f49049c8140fffd0d1e15` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:37.979527 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908b5961-82a4-452a-83a4-4753ebad0cd5 None admin] RBAC: Authorizing `identity:get_project(project_id=c980d1440d8f49049c8140fffd0d1e15)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:37.980922 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908b5961-82a4-452a-83a4-4753ebad0cd5 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:37.982635 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 184/369] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Jul 25 12:01:37 2022] GET /identity/v3/projects/c980d1440d8f49049c8140fffd0d1e15 => generated 302 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:01:37.983395 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908b5961-82a4-452a-83a4-4753ebad0cd5 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:38.066367 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:38.077156 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab4a1cc-0616-46f5-a628-ffd53b11d947 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:38.101510 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908b5961-82a4-452a-83a4-4753ebad0cd5 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:38.115127 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daad4f03-0a62-4527-8afc-4ab46a2c1810 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:38.116116 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daad4f03-0a62-4527-8afc-4ab46a2c1810 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:38.116585 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daad4f03-0a62-4527-8afc-4ab46a2c1810 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:38.117046 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daad4f03-0a62-4527-8afc-4ab46a2c1810 None admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:38.117973 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daad4f03-0a62-4527-8afc-4ab46a2c1810 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:01:38.118485 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daad4f03-0a62-4527-8afc-4ab46a2c1810 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:38.120081 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daad4f03-0a62-4527-8afc-4ab46a2c1810 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:38.129217 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 186/370] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Jul 25 12:01:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:01:38.130350 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:38.130626 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daad4f03-0a62-4527-8afc-4ab46a2c1810 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:38.140183 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549e7821-6331-4111-9311-1c816d646005 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:38.140838 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549e7821-6331-4111-9311-1c816d646005 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:38.141133 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549e7821-6331-4111-9311-1c816d646005 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:38.141407 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549e7821-6331-4111-9311-1c816d646005 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:38.217299 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549e7821-6331-4111-9311-1c816d646005 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:38.218872 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549e7821-6331-4111-9311-1c816d646005 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:38.240049 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daad4f03-0a62-4527-8afc-4ab46a2c1810 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:38.276367 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f274cfd2-f023-4715-9bbe-196262976772 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:38.277539 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f274cfd2-f023-4715-9bbe-196262976772 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:38.278008 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f274cfd2-f023-4715-9bbe-196262976772 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:38.278622 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f274cfd2-f023-4715-9bbe-196262976772 None admin] PATH_INFO: `/v3/groups/be589db33cdd45c3b8f0684e8ac54c2b` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:38.280361 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f274cfd2-f023-4715-9bbe-196262976772 None admin] RBAC: Authorizing `identity:get_group(group_id=be589db33cdd45c3b8f0684e8ac54c2b)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:38.282019 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f274cfd2-f023-4715-9bbe-196262976772 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:38.285659 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 187/371] 10.222.0.42 () {56 vars in 1180 bytes} [Mon Jul 25 12:01:38 2022] GET /identity/v3/groups/be589db33cdd45c3b8f0684e8ac54c2b => generated 226 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:01:38.286988 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f274cfd2-f023-4715-9bbe-196262976772 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:38.314734 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:38.343833 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 185/372] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Jul 25 12:01:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 12:01:38.344695 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549e7821-6331-4111-9311-1c816d646005 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:38.375848 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:38.396559 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f274cfd2-f023-4715-9bbe-196262976772 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:38.432192 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985538a7-6b91-4445-b507-4b7e1cbfae3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:38.433158 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985538a7-6b91-4445-b507-4b7e1cbfae3a None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:38.433630 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985538a7-6b91-4445-b507-4b7e1cbfae3a None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:38.434090 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985538a7-6b91-4445-b507-4b7e1cbfae3a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:38.456064 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549e7821-6331-4111-9311-1c816d646005 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:38.493932 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07b4b2c-4843-4178-9e46-2d6b258f4ef1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:38.494446 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07b4b2c-4843-4178-9e46-2d6b258f4ef1 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:38.494734 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07b4b2c-4843-4178-9e46-2d6b258f4ef1 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:38.495012 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07b4b2c-4843-4178-9e46-2d6b258f4ef1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:38.495915 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07b4b2c-4843-4178-9e46-2d6b258f4ef1 None admin] RBAC: Adding query filter params (group.id=be589db33cdd45c3b8f0684e8ac54c2b, role.id=472313ae69174578824df0bed6390744, scope.project.id=c980d1440d8f49049c8140fffd0d1e15) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:01:38.496262 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07b4b2c-4843-4178-9e46-2d6b258f4ef1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:38.497964 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07b4b2c-4843-4178-9e46-2d6b258f4ef1 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:38.505135 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 186/373] 10.222.0.42 () {56 vars in 1376 bytes} [Mon Jul 25 12:01:38 2022] GET /identity/v3/role_assignments?group.id=be589db33cdd45c3b8f0684e8ac54c2b&scope.project.id=c980d1440d8f49049c8140fffd0d1e15&role.id=472313ae69174578824df0bed6390744 => generated 260 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:01:38.506371 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07b4b2c-4843-4178-9e46-2d6b258f4ef1 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:38.508611 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985538a7-6b91-4445-b507-4b7e1cbfae3a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:38.510346 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985538a7-6b91-4445-b507-4b7e1cbfae3a None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:38.539422 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:38.639377 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 188/374] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Jul 25 12:01:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 12:01:38.639734 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:38.648022 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985538a7-6b91-4445-b507-4b7e1cbfae3a None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:38.652578 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07b4b2c-4843-4178-9e46-2d6b258f4ef1 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:38.694426 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc2149f-e230-42f2-9900-4dd6fe592806 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:38.695875 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc2149f-e230-42f2-9900-4dd6fe592806 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:38.696507 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc2149f-e230-42f2-9900-4dd6fe592806 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:38.696991 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc2149f-e230-42f2-9900-4dd6fe592806 None admin] PATH_INFO: `/v3/projects/c980d1440d8f49049c8140fffd0d1e15` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:38.698692 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc2149f-e230-42f2-9900-4dd6fe592806 None admin] RBAC: Authorizing `identity:get_project(project_id=c980d1440d8f49049c8140fffd0d1e15)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:38.700552 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc2149f-e230-42f2-9900-4dd6fe592806 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:38.702647 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 187/375] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Jul 25 12:01:38 2022] GET /identity/v3/projects/c980d1440d8f49049c8140fffd0d1e15 => generated 302 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:01:38.708352 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc2149f-e230-42f2-9900-4dd6fe592806 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:38.775434 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985538a7-6b91-4445-b507-4b7e1cbfae3a None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:38.800602 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:38.813221 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e414002d-7a19-4f01-8f7e-4f7253ac577a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:38.814175 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e414002d-7a19-4f01-8f7e-4f7253ac577a None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:38.814674 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e414002d-7a19-4f01-8f7e-4f7253ac577a None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:38.815141 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e414002d-7a19-4f01-8f7e-4f7253ac577a None admin] PATH_INFO: `/v3/projects/827585a7cfe8411987f9b81102c1c103` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:38.816635 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e414002d-7a19-4f01-8f7e-4f7253ac577a None admin] RBAC: Authorizing `identity:get_project(project_id=827585a7cfe8411987f9b81102c1c103)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:38.818147 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e414002d-7a19-4f01-8f7e-4f7253ac577a None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:38.820302 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 189/376] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Jul 25 12:01:38 2022] GET /identity/v3/projects/827585a7cfe8411987f9b81102c1c103 => generated 344 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:01:38.825237 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e414002d-7a19-4f01-8f7e-4f7253ac577a None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:38.832068 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc2149f-e230-42f2-9900-4dd6fe592806 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:38.928833 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cb8d68-24a4-4168-85cc-7ceeac1e6ef4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:38.929724 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cb8d68-24a4-4168-85cc-7ceeac1e6ef4 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:38.930012 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cb8d68-24a4-4168-85cc-7ceeac1e6ef4 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:38.930295 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cb8d68-24a4-4168-85cc-7ceeac1e6ef4 None admin] PATH_INFO: `/v3/groups/be589db33cdd45c3b8f0684e8ac54c2b` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:38.936282 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50cb8d68-24a4-4168-85cc-7ceeac1e6ef4 None admin] RBAC: Authorizing `identity:get_group(group_id=be589db33cdd45c3b8f0684e8ac54c2b)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:38.937751 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50cb8d68-24a4-4168-85cc-7ceeac1e6ef4 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:38.939786 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 188/377] 10.222.0.42 () {56 vars in 1180 bytes} [Mon Jul 25 12:01:38 2022] GET /identity/v3/groups/be589db33cdd45c3b8f0684e8ac54c2b => generated 226 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:01:38.945438 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cb8d68-24a4-4168-85cc-7ceeac1e6ef4 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:38.952123 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e414002d-7a19-4f01-8f7e-4f7253ac577a None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:38.991578 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:38.991808 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37156170-a222-4ffa-a414-a310bce6d2f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:38.993654 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37156170-a222-4ffa-a414-a310bce6d2f2 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:38.994091 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37156170-a222-4ffa-a414-a310bce6d2f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:38.994654 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37156170-a222-4ffa-a414-a310bce6d2f2 None admin] PATH_INFO: `/v3/groups/ce014e9e4c2c4a64994b15e0fe2ae52a` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:39.002843 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37156170-a222-4ffa-a414-a310bce6d2f2 None admin] RBAC: Authorizing `identity:get_group(group_id=ce014e9e4c2c4a64994b15e0fe2ae52a)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:39.004563 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37156170-a222-4ffa-a414-a310bce6d2f2 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:39.007034 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 190/378] 10.222.0.42 () {56 vars in 1180 bytes} [Mon Jul 25 12:01:38 2022] GET /identity/v3/groups/ce014e9e4c2c4a64994b15e0fe2ae52a => generated 229 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 12:01:39.012669 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37156170-a222-4ffa-a414-a310bce6d2f2 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:39.082812 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cb8d68-24a4-4168-85cc-7ceeac1e6ef4 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:39.118231 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:39.119039 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82961fda-366c-47ca-89f8-d67998636bd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:39.119743 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82961fda-366c-47ca-89f8-d67998636bd2 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:39.120057 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82961fda-366c-47ca-89f8-d67998636bd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:39.120338 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82961fda-366c-47ca-89f8-d67998636bd2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:39.121187 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82961fda-366c-47ca-89f8-d67998636bd2 None admin] RBAC: Adding query filter params (group.id=be589db33cdd45c3b8f0684e8ac54c2b, role.id=dea3ef72822043f3baf3bcc1be7c318f, scope.project.id=c980d1440d8f49049c8140fffd0d1e15) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:01:39.121508 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82961fda-366c-47ca-89f8-d67998636bd2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:39.122998 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82961fda-366c-47ca-89f8-d67998636bd2 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:39.129339 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 189/379] 10.222.0.42 () {56 vars in 1376 bytes} [Mon Jul 25 12:01:38 2022] GET /identity/v3/role_assignments?group.id=be589db33cdd45c3b8f0684e8ac54c2b&scope.project.id=c980d1440d8f49049c8140fffd0d1e15&role.id=dea3ef72822043f3baf3bcc1be7c318f => generated 260 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 12:01:39.186081 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37156170-a222-4ffa-a414-a310bce6d2f2 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:39.224456 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30cf9bb-fe57-4c1d-9921-a58f6e9fd0c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:39.225145 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30cf9bb-fe57-4c1d-9921-a58f6e9fd0c5 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:39.225442 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30cf9bb-fe57-4c1d-9921-a58f6e9fd0c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:39.225730 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30cf9bb-fe57-4c1d-9921-a58f6e9fd0c5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:39.226578 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30cf9bb-fe57-4c1d-9921-a58f6e9fd0c5 None admin] RBAC: Adding query filter params (group.id=ce014e9e4c2c4a64994b15e0fe2ae52a, role.id=a7d20885b6d94928a34b3cc5b3a00006, scope.project.id=827585a7cfe8411987f9b81102c1c103) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:01:39.226898 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30cf9bb-fe57-4c1d-9921-a58f6e9fd0c5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:39.228393 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30cf9bb-fe57-4c1d-9921-a58f6e9fd0c5 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:39.234920 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 191/380] 10.222.0.42 () {56 vars in 1376 bytes} [Mon Jul 25 12:01:39 2022] GET /identity/v3/role_assignments?group.id=ce014e9e4c2c4a64994b15e0fe2ae52a&scope.project.id=827585a7cfe8411987f9b81102c1c103&role.id=a7d20885b6d94928a34b3cc5b3a00006 => generated 260 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 12:01:39.298294 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5fd839-e3d6-47aa-bbd7-0dcd7d26924b None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:39.298626 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5fd839-e3d6-47aa-bbd7-0dcd7d26924b None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:39.298864 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5fd839-e3d6-47aa-bbd7-0dcd7d26924b None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:39.300338 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 190/381] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 25 12:01:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:01:39.306027 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a914bd-6297-4aab-bbe3-2fa9c808e301 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:39.306536 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a914bd-6297-4aab-bbe3-2fa9c808e301 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:39.306990 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a914bd-6297-4aab-bbe3-2fa9c808e301 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:39.331966 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-97a914bd-6297-4aab-bbe3-2fa9c808e301 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:01:39.333543 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-97a914bd-6297-4aab-bbe3-2fa9c808e301 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:01:39.421742 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 192/382] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 25 12:01:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 12:01:39.445912 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5fd839-e3d6-47aa-bbd7-0dcd7d26924b None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:39.559386 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5fd839-e3d6-47aa-bbd7-0dcd7d26924b None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:39.562260 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:39.597665 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88034ffd-73b5-4b50-9d4a-a99d789bcfb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:39.598407 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88034ffd-73b5-4b50-9d4a-a99d789bcfb3 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:39.598697 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88034ffd-73b5-4b50-9d4a-a99d789bcfb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:39.598980 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88034ffd-73b5-4b50-9d4a-a99d789bcfb3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:39.609929 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88034ffd-73b5-4b50-9d4a-a99d789bcfb3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 12:01:39.617830 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-88034ffd-73b5-4b50-9d4a-a99d789bcfb3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 25 12:01:39.617830 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:01:39.617830 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:01:39.617830 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:01:39.617830 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:01:39.617830 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:01:39.617830 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:01:39.617830 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:01:39.617830 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:01:39.617830 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:01:39.617830 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:01:39.617830 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:01:39.617830 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 12:01:39.617830 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 12:01:39.617830 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 12:01:39.617830 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 12:01:39.618706 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 12:01:39.618706 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 12:01:39.618706 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 12:01:39.618706 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 12:01:39.618706 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:01:39.618706 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:01:39.618706 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:01:39.618706 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:01:39.618706 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:01:39.618706 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:01:39.618706 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:01:39.618706 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 25 12:01:39.618706 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:01:39.618706 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:01:39.618706 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:01:39.618706 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:01:39.619498 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:01:39.619498 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:01:39.619498 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:01:39.619498 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:01:39.619498 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 12:01:39.619498 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 12:01:39.619498 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 12:01:39.619498 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 12:01:39.619498 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 12:01:39.619498 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 12:01:39.619498 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 25 12:01:39.619498 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 25 12:01:39.619498 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 191/383] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Jul 25 12:01:39 2022] GET /identity/v3/roles/member => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:01:39.625204 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a914bd-6297-4aab-bbe3-2fa9c808e301 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:39.683659 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:39.746621 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a914bd-6297-4aab-bbe3-2fa9c808e301 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:39.788990 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68244a1-ef34-4669-a6fc-f8f504fcdb3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:39.789687 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68244a1-ef34-4669-a6fc-f8f504fcdb3f None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:39.789978 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68244a1-ef34-4669-a6fc-f8f504fcdb3f None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:39.790260 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68244a1-ef34-4669-a6fc-f8f504fcdb3f None admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:39.791279 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68244a1-ef34-4669-a6fc-f8f504fcdb3f None admin] RBAC: Adding query filter params (name=member) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:01:39.791611 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68244a1-ef34-4669-a6fc-f8f504fcdb3f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:39.793006 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68244a1-ef34-4669-a6fc-f8f504fcdb3f None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:39.802928 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 193/384] 10.222.0.42 () {56 vars in 1101 bytes} [Mon Jul 25 12:01:39 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:01:39.808509 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88034ffd-73b5-4b50-9d4a-a99d789bcfb3 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:39.920903 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88034ffd-73b5-4b50-9d4a-a99d789bcfb3 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:39.956234 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46253afc-9d2e-45f3-96c4-6ef23b1b8e14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:39.957107 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46253afc-9d2e-45f3-96c4-6ef23b1b8e14 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:39.957398 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46253afc-9d2e-45f3-96c4-6ef23b1b8e14 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:39.957680 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46253afc-9d2e-45f3-96c4-6ef23b1b8e14 None admin] PATH_INFO: `/v3/groups/be589db33cdd45c3b8f0684e8ac54c2b` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:39.958879 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46253afc-9d2e-45f3-96c4-6ef23b1b8e14 None admin] RBAC: Authorizing `identity:get_group(group_id=be589db33cdd45c3b8f0684e8ac54c2b)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:39.960383 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46253afc-9d2e-45f3-96c4-6ef23b1b8e14 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:39.962310 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 192/385] 10.222.0.42 () {56 vars in 1180 bytes} [Mon Jul 25 12:01:39 2022] GET /identity/v3/groups/be589db33cdd45c3b8f0684e8ac54c2b => generated 226 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:01:39.967635 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46253afc-9d2e-45f3-96c4-6ef23b1b8e14 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:39.968408 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9deb5b-8445-49c1-9efa-0a3ca2098e53 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:39.969214 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9deb5b-8445-49c1-9efa-0a3ca2098e53 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:39.969686 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9deb5b-8445-49c1-9efa-0a3ca2098e53 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:39.975637 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 194/386] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 25 12:01:39 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 12:01:39.977114 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae832d8-22b5-463f-b197-20b37eedc1ad None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:39.977596 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae832d8-22b5-463f-b197-20b37eedc1ad None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:39.978068 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae832d8-22b5-463f-b197-20b37eedc1ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:39.991908 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:40.020546 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-7ae832d8-22b5-463f-b197-20b37eedc1ad None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:01:40.022076 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-7ae832d8-22b5-463f-b197-20b37eedc1ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:01:40.081059 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46253afc-9d2e-45f3-96c4-6ef23b1b8e14 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:40.119470 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c846d84e-5d93-4715-aee4-dc24291070c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:40.120425 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c846d84e-5d93-4715-aee4-dc24291070c5 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:40.120860 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c846d84e-5d93-4715-aee4-dc24291070c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:40.121170 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c846d84e-5d93-4715-aee4-dc24291070c5 None admin] PATH_INFO: `/v3/projects/23c9d26ac78646c79204b87bd5332e2c` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:40.122611 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c846d84e-5d93-4715-aee4-dc24291070c5 None admin] RBAC: Authorizing `identity:get_project(project_id=23c9d26ac78646c79204b87bd5332e2c)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:40.124344 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c846d84e-5d93-4715-aee4-dc24291070c5 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:40.126322 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 193/387] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Jul 25 12:01:39 2022] GET /identity/v3/projects/23c9d26ac78646c79204b87bd5332e2c => generated 298 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 12:01:40.132597 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c846d84e-5d93-4715-aee4-dc24291070c5 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:40.209041 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 195/388] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 25 12:01:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 12:01:40.231633 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae832d8-22b5-463f-b197-20b37eedc1ad None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:40.307934 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:40.314559 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:40.335927 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c846d84e-5d93-4715-aee4-dc24291070c5 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:40.382547 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a44c95-8244-4c80-9037-5cb429a72e40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:40.383467 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a44c95-8244-4c80-9037-5cb429a72e40 None admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:40.384045 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a44c95-8244-4c80-9037-5cb429a72e40 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:40.384338 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a44c95-8244-4c80-9037-5cb429a72e40 None admin] PATH_INFO: `/v3/projects/23c9d26ac78646c79204b87bd5332e2c/groups/be589db33cdd45c3b8f0684e8ac54c2b/roles/472313ae69174578824df0bed6390744` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:40.386477 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a44c95-8244-4c80-9037-5cb429a72e40 None admin] RBAC: Authorizing `identity:create_grant(project_id=23c9d26ac78646c79204b87bd5332e2c, group_id=be589db33cdd45c3b8f0684e8ac54c2b, role_id=472313ae69174578824df0bed6390744)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:40.388406 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a44c95-8244-4c80-9037-5cb429a72e40 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:40.414406 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 194/389] 10.222.0.42 () {58 vars in 1442 bytes} [Mon Jul 25 12:01:40 2022] PUT /identity/v3/projects/23c9d26ac78646c79204b87bd5332e2c/groups/be589db33cdd45c3b8f0684e8ac54c2b/roles/472313ae69174578824df0bed6390744 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:01:40.418518 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae832d8-22b5-463f-b197-20b37eedc1ad None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:40.464294 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3438ba42-107f-4168-b7ea-64676ddbadef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:40.465141 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3438ba42-107f-4168-b7ea-64676ddbadef None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:40.465436 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3438ba42-107f-4168-b7ea-64676ddbadef None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:40.465723 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3438ba42-107f-4168-b7ea-64676ddbadef None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:40.481423 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3438ba42-107f-4168-b7ea-64676ddbadef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 12:01:40.492968 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-3438ba42-107f-4168-b7ea-64676ddbadef None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 12:01:40.492968 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:01:40.492968 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:01:40.492968 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:01:40.492968 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:01:40.492968 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:01:40.492968 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:01:40.492968 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:01:40.492968 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:01:40.492968 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:01:40.492968 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:01:40.492968 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:01:40.492968 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 12:01:40.492968 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 12:01:40.492968 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 12:01:40.492968 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 12:01:40.493920 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 12:01:40.493920 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 12:01:40.493920 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 12:01:40.493920 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 12:01:40.493920 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:01:40.493920 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:01:40.493920 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:01:40.493920 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:01:40.493920 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:01:40.493920 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:01:40.493920 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:01:40.493920 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jul 25 12:01:40.493920 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:01:40.493920 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:01:40.493920 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:01:40.493920 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:01:40.494722 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:01:40.494722 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:01:40.494722 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:01:40.494722 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:01:40.494722 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 12:01:40.494722 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 12:01:40.494722 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 12:01:40.494722 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 12:01:40.494722 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 12:01:40.494722 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 12:01:40.494722 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 12:01:40.494722 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jul 25 12:01:40.494722 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 196/390] 10.222.0.42 () {56 vars in 1114 bytes} [Mon Jul 25 12:01:40 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:01:40.500453 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a44c95-8244-4c80-9037-5cb429a72e40 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:40.627758 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a44c95-8244-4c80-9037-5cb429a72e40 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:40.675268 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc05b0c1-a8c2-4289-ab28-e6d150573836 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:40.675998 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc05b0c1-a8c2-4289-ab28-e6d150573836 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:40.676292 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc05b0c1-a8c2-4289-ab28-e6d150573836 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:40.676574 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc05b0c1-a8c2-4289-ab28-e6d150573836 None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:40.677354 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc05b0c1-a8c2-4289-ab28-e6d150573836 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:01:40.677673 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc05b0c1-a8c2-4289-ab28-e6d150573836 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:40.679083 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc05b0c1-a8c2-4289-ab28-e6d150573836 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:40.691775 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 195/391] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Jul 25 12:01:40 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:01:40.699599 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3438ba42-107f-4168-b7ea-64676ddbadef None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:40.815468 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3438ba42-107f-4168-b7ea-64676ddbadef None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:40.817181 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:40.852958 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04347b7d-a816-4565-be8c-ae3fa64c8ac0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:40.853827 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04347b7d-a816-4565-be8c-ae3fa64c8ac0 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:40.854112 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04347b7d-a816-4565-be8c-ae3fa64c8ac0 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:40.854395 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04347b7d-a816-4565-be8c-ae3fa64c8ac0 None admin] PATH_INFO: `/v3/groups/be589db33cdd45c3b8f0684e8ac54c2b` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:40.855613 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04347b7d-a816-4565-be8c-ae3fa64c8ac0 None admin] RBAC: Authorizing `identity:get_group(group_id=be589db33cdd45c3b8f0684e8ac54c2b)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:40.857124 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04347b7d-a816-4565-be8c-ae3fa64c8ac0 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:40.859068 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 197/392] 10.222.0.42 () {56 vars in 1180 bytes} [Mon Jul 25 12:01:40 2022] GET /identity/v3/groups/be589db33cdd45c3b8f0684e8ac54c2b => generated 226 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:01:40.864501 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc05b0c1-a8c2-4289-ab28-e6d150573836 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:40.881085 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:40.987844 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc05b0c1-a8c2-4289-ab28-e6d150573836 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:41.029987 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd272675-2e08-4930-a768-808e771cb213 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:41.031321 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd272675-2e08-4930-a768-808e771cb213 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:41.031885 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd272675-2e08-4930-a768-808e771cb213 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:41.032450 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd272675-2e08-4930-a768-808e771cb213 None admin] PATH_INFO: `/v3/projects/23c9d26ac78646c79204b87bd5332e2c` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:41.034121 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd272675-2e08-4930-a768-808e771cb213 None admin] RBAC: Authorizing `identity:get_project(project_id=23c9d26ac78646c79204b87bd5332e2c)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:41.036779 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd272675-2e08-4930-a768-808e771cb213 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:41.039409 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 196/393] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Jul 25 12:01:40 2022] GET /identity/v3/projects/23c9d26ac78646c79204b87bd5332e2c => generated 298 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:01:41.045404 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04347b7d-a816-4565-be8c-ae3fa64c8ac0 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:41.154419 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04347b7d-a816-4565-be8c-ae3fa64c8ac0 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:41.190867 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ee7a6c-f0a7-48de-b49f-78260889ff84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:41.191790 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ee7a6c-f0a7-48de-b49f-78260889ff84 None admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:41.192218 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ee7a6c-f0a7-48de-b49f-78260889ff84 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:41.192502 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ee7a6c-f0a7-48de-b49f-78260889ff84 None admin] PATH_INFO: `/v3/projects/23c9d26ac78646c79204b87bd5332e2c/groups/be589db33cdd45c3b8f0684e8ac54c2b/roles/dea3ef72822043f3baf3bcc1be7c318f` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:41.194772 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ee7a6c-f0a7-48de-b49f-78260889ff84 None admin] RBAC: Authorizing `identity:create_grant(project_id=23c9d26ac78646c79204b87bd5332e2c, group_id=be589db33cdd45c3b8f0684e8ac54c2b, role_id=dea3ef72822043f3baf3bcc1be7c318f)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:41.196278 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ee7a6c-f0a7-48de-b49f-78260889ff84 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:41.213640 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 198/394] 10.222.0.42 () {58 vars in 1442 bytes} [Mon Jul 25 12:01:41 2022] PUT /identity/v3/projects/23c9d26ac78646c79204b87bd5332e2c/groups/be589db33cdd45c3b8f0684e8ac54c2b/roles/dea3ef72822043f3baf3bcc1be7c318f => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:01:41.258568 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4840258-6d10-4196-9756-6c4413f749b9 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:41.258888 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4840258-6d10-4196-9756-6c4413f749b9 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:41.259135 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4840258-6d10-4196-9756-6c4413f749b9 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:41.261056 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 197/395] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 25 12:01:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:01:41.267265 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744786fc-b106-45be-9584-8191edc55f45 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:41.267595 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744786fc-b106-45be-9584-8191edc55f45 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:41.267939 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744786fc-b106-45be-9584-8191edc55f45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:41.290994 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-744786fc-b106-45be-9584-8191edc55f45 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:01:41.292238 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-744786fc-b106-45be-9584-8191edc55f45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:01:41.327284 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:41.388744 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 199/396] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 25 12:01:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 12:01:41.411579 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4840258-6d10-4196-9756-6c4413f749b9 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:41.507472 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:41.536050 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4840258-6d10-4196-9756-6c4413f749b9 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:41.577942 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4bb44b-72ba-4519-bb1a-6789bfcfec4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:41.578812 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4bb44b-72ba-4519-bb1a-6789bfcfec4d None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:41.579108 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4bb44b-72ba-4519-bb1a-6789bfcfec4d None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:41.579433 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4bb44b-72ba-4519-bb1a-6789bfcfec4d None admin] PATH_INFO: `/v3/roles/member` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:41.593206 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4bb44b-72ba-4519-bb1a-6789bfcfec4d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 12:01:41.601301 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-bb4bb44b-72ba-4519-bb1a-6789bfcfec4d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 25 12:01:41.601301 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:01:41.601301 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:01:41.601301 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:01:41.601301 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:01:41.601301 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:01:41.601301 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:01:41.601301 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:01:41.601301 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:01:41.601301 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:01:41.601301 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:01:41.601301 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:01:41.601301 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 12:01:41.601301 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 12:01:41.601301 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 12:01:41.601301 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 12:01:41.603437 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 12:01:41.603437 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 12:01:41.603437 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 12:01:41.603437 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 12:01:41.603437 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:01:41.603437 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:01:41.603437 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:01:41.603437 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:01:41.603437 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:01:41.603437 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:01:41.603437 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:01:41.603437 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 25 12:01:41.603437 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:01:41.603437 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:01:41.603437 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:01:41.603437 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:01:41.605439 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:01:41.605439 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:01:41.605439 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:01:41.605439 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:01:41.605439 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 12:01:41.605439 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 12:01:41.605439 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 12:01:41.605439 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 12:01:41.605439 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 12:01:41.605439 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 12:01:41.605439 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 25 12:01:41.605439 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 25 12:01:41.605439 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 198/397] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Jul 25 12:01:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:01:41.610288 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744786fc-b106-45be-9584-8191edc55f45 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:41.718870 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744786fc-b106-45be-9584-8191edc55f45 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:41.756238 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8912149-6a2d-47c4-b58c-663bb61200bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:41.756922 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8912149-6a2d-47c4-b58c-663bb61200bf None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:41.757215 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8912149-6a2d-47c4-b58c-663bb61200bf None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:41.757499 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8912149-6a2d-47c4-b58c-663bb61200bf None admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:41.758513 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8912149-6a2d-47c4-b58c-663bb61200bf None admin] RBAC: Adding query filter params (name=member) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:01:41.758869 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8912149-6a2d-47c4-b58c-663bb61200bf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:41.760551 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8912149-6a2d-47c4-b58c-663bb61200bf None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:41.771282 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 200/398] 10.222.0.42 () {56 vars in 1101 bytes} [Mon Jul 25 12:01:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:01:41.780170 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4bb44b-72ba-4519-bb1a-6789bfcfec4d None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:41.893967 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4bb44b-72ba-4519-bb1a-6789bfcfec4d None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:41.929590 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e39dc9a-36f2-4c75-a697-c760fc818a8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:41.930482 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e39dc9a-36f2-4c75-a697-c760fc818a8c None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:41.930773 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e39dc9a-36f2-4c75-a697-c760fc818a8c None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:41.931056 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e39dc9a-36f2-4c75-a697-c760fc818a8c None admin] PATH_INFO: `/v3/groups/be589db33cdd45c3b8f0684e8ac54c2b` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:41.932283 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e39dc9a-36f2-4c75-a697-c760fc818a8c None admin] RBAC: Authorizing `identity:get_group(group_id=be589db33cdd45c3b8f0684e8ac54c2b)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:41.933750 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e39dc9a-36f2-4c75-a697-c760fc818a8c None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:41.935696 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 199/399] 10.222.0.42 () {56 vars in 1180 bytes} [Mon Jul 25 12:01:41 2022] GET /identity/v3/groups/be589db33cdd45c3b8f0684e8ac54c2b => generated 226 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:01:41.944684 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8912149-6a2d-47c4-b58c-663bb61200bf None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:41.970535 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:42.016927 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005d4143-ee91-4b5d-900f-b29b8b5c0603 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:42.017427 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005d4143-ee91-4b5d-900f-b29b8b5c0603 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:42.017513 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005d4143-ee91-4b5d-900f-b29b8b5c0603 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:42.019791 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 200/400] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 25 12:01:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:01:42.025600 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b766dc-6009-4cb0-a938-f1509d8aa5e7 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:42.025903 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b766dc-6009-4cb0-a938-f1509d8aa5e7 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:42.027016 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b766dc-6009-4cb0-a938-f1509d8aa5e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:42.043369 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:42.049994 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-46b766dc-6009-4cb0-a938-f1509d8aa5e7 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:01:42.051512 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-46b766dc-6009-4cb0-a938-f1509d8aa5e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:01:42.086227 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8912149-6a2d-47c4-b58c-663bb61200bf None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:42.133825 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b289975c-a38e-4183-90fa-3f0d0b882367 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:42.135052 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b289975c-a38e-4183-90fa-3f0d0b882367 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:42.135582 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b289975c-a38e-4183-90fa-3f0d0b882367 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:42.136073 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b289975c-a38e-4183-90fa-3f0d0b882367 None admin] PATH_INFO: `/v3/projects/c980d1440d8f49049c8140fffd0d1e15` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:42.136532 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 201/401] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 25 12:01:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 12:01:42.137746 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b289975c-a38e-4183-90fa-3f0d0b882367 None admin] RBAC: Authorizing `identity:get_project(project_id=c980d1440d8f49049c8140fffd0d1e15)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:42.139410 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b289975c-a38e-4183-90fa-3f0d0b882367 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:42.142148 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 201/402] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Jul 25 12:01:41 2022] GET /identity/v3/projects/c980d1440d8f49049c8140fffd0d1e15 => generated 302 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:01:42.147775 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b766dc-6009-4cb0-a938-f1509d8aa5e7 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:42.179973 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b289975c-a38e-4183-90fa-3f0d0b882367 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:42.255104 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b766dc-6009-4cb0-a938-f1509d8aa5e7 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:42.301268 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8158e3d-e89f-4ad4-9ea1-bcf2efd78b3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:42.302200 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8158e3d-e89f-4ad4-9ea1-bcf2efd78b3e None admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:42.302475 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8158e3d-e89f-4ad4-9ea1-bcf2efd78b3e None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:42.302762 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8158e3d-e89f-4ad4-9ea1-bcf2efd78b3e None admin] PATH_INFO: `/v3/projects/c980d1440d8f49049c8140fffd0d1e15/groups/be589db33cdd45c3b8f0684e8ac54c2b/roles/472313ae69174578824df0bed6390744` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:42.304816 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8158e3d-e89f-4ad4-9ea1-bcf2efd78b3e None admin] RBAC: Authorizing `identity:create_grant(project_id=c980d1440d8f49049c8140fffd0d1e15, group_id=be589db33cdd45c3b8f0684e8ac54c2b, role_id=472313ae69174578824df0bed6390744)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:42.306365 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8158e3d-e89f-4ad4-9ea1-bcf2efd78b3e None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:42.309728 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:42.314285 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:42.334786 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 202/403] 10.222.0.42 () {58 vars in 1442 bytes} [Mon Jul 25 12:01:42 2022] PUT /identity/v3/projects/c980d1440d8f49049c8140fffd0d1e15/groups/be589db33cdd45c3b8f0684e8ac54c2b/roles/472313ae69174578824df0bed6390744 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:01:42.338300 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b289975c-a38e-4183-90fa-3f0d0b882367 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:42.372478 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3e455f-c401-4336-a679-616585d60259 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:42.372954 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3e455f-c401-4336-a679-616585d60259 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:42.373246 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3e455f-c401-4336-a679-616585d60259 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:42.374961 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 203/404] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 25 12:01:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 12:01:42.380286 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee4c0bf-0877-4020-9d12-9e71252020ec None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:42.380582 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee4c0bf-0877-4020-9d12-9e71252020ec None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:42.380872 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee4c0bf-0877-4020-9d12-9e71252020ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:42.382670 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ead9376-b64b-427e-873c-5e73af7202c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:42.383503 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ead9376-b64b-427e-873c-5e73af7202c2 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:42.383821 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ead9376-b64b-427e-873c-5e73af7202c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:42.384148 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ead9376-b64b-427e-873c-5e73af7202c2 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:42.404294 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ead9376-b64b-427e-873c-5e73af7202c2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 12:01:42.410261 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-6ee4c0bf-0877-4020-9d12-9e71252020ec None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:01:42.411492 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-6ee4c0bf-0877-4020-9d12-9e71252020ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:01:42.417909 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-7ead9376-b64b-427e-873c-5e73af7202c2 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 12:01:42.417909 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:01:42.417909 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:01:42.417909 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:01:42.417909 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:01:42.417909 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:01:42.417909 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:01:42.417909 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:01:42.417909 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:01:42.417909 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:01:42.417909 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:01:42.417909 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:01:42.417909 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 12:01:42.417909 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 12:01:42.417909 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 12:01:42.417909 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 12:01:42.420179 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 12:01:42.420179 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 12:01:42.420179 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 12:01:42.420179 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 12:01:42.420179 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:01:42.420179 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:01:42.420179 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:01:42.420179 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:01:42.420179 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:01:42.420179 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:01:42.420179 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:01:42.420179 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jul 25 12:01:42.420179 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:01:42.420179 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:01:42.420179 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:01:42.420179 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:01:42.422436 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:01:42.422436 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:01:42.422436 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:01:42.422436 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:01:42.422436 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 12:01:42.422436 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 12:01:42.422436 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 12:01:42.422436 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 12:01:42.422436 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 12:01:42.422436 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 12:01:42.422436 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 12:01:42.422436 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jul 25 12:01:42.422436 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 202/405] 10.222.0.42 () {56 vars in 1114 bytes} [Mon Jul 25 12:01:42 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 12:01:42.430148 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ead9376-b64b-427e-873c-5e73af7202c2 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:42.546271 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 204/406] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 25 12:01:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 25 12:01:42.574328 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee4c0bf-0877-4020-9d12-9e71252020ec None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:42.590693 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ead9376-b64b-427e-873c-5e73af7202c2 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:42.631652 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f158b3-0827-4471-a6d5-752e97284153 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:42.632158 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f158b3-0827-4471-a6d5-752e97284153 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:42.632457 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f158b3-0827-4471-a6d5-752e97284153 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:42.632752 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f158b3-0827-4471-a6d5-752e97284153 None admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:42.633577 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f158b3-0827-4471-a6d5-752e97284153 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:01:42.633835 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f158b3-0827-4471-a6d5-752e97284153 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:42.635349 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f158b3-0827-4471-a6d5-752e97284153 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:42.648455 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 203/407] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Jul 25 12:01:42 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 12:01:42.653588 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f158b3-0827-4471-a6d5-752e97284153 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:42.676032 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:42.692223 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee4c0bf-0877-4020-9d12-9e71252020ec None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:42.741820 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:42.759137 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac73a91-de15-4ba9-9f61-6e3194c7d037 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:42.759993 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac73a91-de15-4ba9-9f61-6e3194c7d037 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:42.760282 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac73a91-de15-4ba9-9f61-6e3194c7d037 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:42.760563 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac73a91-de15-4ba9-9f61-6e3194c7d037 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:42.788689 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac73a91-de15-4ba9-9f61-6e3194c7d037 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 12:01:42.799332 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-9ac73a91-de15-4ba9-9f61-6e3194c7d037 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 12:01:42.799332 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:01:42.799332 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:01:42.799332 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:01:42.799332 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:01:42.799332 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:01:42.799332 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:01:42.799332 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:01:42.799332 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:01:42.799332 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:01:42.799332 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:01:42.799332 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:01:42.799332 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 12:01:42.799332 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 12:01:42.799332 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 12:01:42.799332 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 12:01:42.800276 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 12:01:42.800276 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 12:01:42.800276 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 12:01:42.800276 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 12:01:42.800276 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:01:42.800276 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:01:42.800276 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:01:42.800276 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:01:42.800276 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:01:42.800276 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:01:42.800276 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:01:42.800276 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 25 12:01:42.800276 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:01:42.800276 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:01:42.800276 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:01:42.800276 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:01:42.801071 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:01:42.801071 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:01:42.801071 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:01:42.801071 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:01:42.801071 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 12:01:42.801071 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 12:01:42.801071 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 12:01:42.801071 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 12:01:42.801071 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 12:01:42.801071 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 12:01:42.801071 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 12:01:42.801071 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 25 12:01:42.801867 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 205/408] 10.222.0.42 () {56 vars in 1096 bytes} [Mon Jul 25 12:01:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 12:01:42.807737 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac73a91-de15-4ba9-9f61-6e3194c7d037 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:42.845488 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f158b3-0827-4471-a6d5-752e97284153 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:42.918016 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc64a0a-e4c4-405e-9244-5c3f109166dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:42.918894 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc64a0a-e4c4-405e-9244-5c3f109166dc None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:42.919182 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc64a0a-e4c4-405e-9244-5c3f109166dc None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:42.919463 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc64a0a-e4c4-405e-9244-5c3f109166dc None admin] PATH_INFO: `/v3/groups/be589db33cdd45c3b8f0684e8ac54c2b` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:42.923421 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc64a0a-e4c4-405e-9244-5c3f109166dc None admin] RBAC: Authorizing `identity:get_group(group_id=be589db33cdd45c3b8f0684e8ac54c2b)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:42.924922 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc64a0a-e4c4-405e-9244-5c3f109166dc None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:42.928002 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 204/409] 10.222.0.42 () {56 vars in 1180 bytes} [Mon Jul 25 12:01:42 2022] GET /identity/v3/groups/be589db33cdd45c3b8f0684e8ac54c2b => generated 226 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:01:42.932916 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc64a0a-e4c4-405e-9244-5c3f109166dc None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:42.992457 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac73a91-de15-4ba9-9f61-6e3194c7d037 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:43.052780 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec46c04-5313-45db-964a-53e8b764370a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:43.054314 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec46c04-5313-45db-964a-53e8b764370a None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:43.054794 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec46c04-5313-45db-964a-53e8b764370a None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:43.055281 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec46c04-5313-45db-964a-53e8b764370a None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:43.056357 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec46c04-5313-45db-964a-53e8b764370a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:01:43.056883 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec46c04-5313-45db-964a-53e8b764370a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:43.058473 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec46c04-5313-45db-964a-53e8b764370a None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:43.071672 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 206/410] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Jul 25 12:01:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:01:43.072263 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:43.075104 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec46c04-5313-45db-964a-53e8b764370a None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:43.091921 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc64a0a-e4c4-405e-9244-5c3f109166dc None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:43.127677 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cde753f-dda7-4ad4-89f1-e6a0281cfa66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:43.128475 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cde753f-dda7-4ad4-89f1-e6a0281cfa66 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:43.128808 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cde753f-dda7-4ad4-89f1-e6a0281cfa66 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:43.129096 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cde753f-dda7-4ad4-89f1-e6a0281cfa66 None admin] PATH_INFO: `/v3/projects/c980d1440d8f49049c8140fffd0d1e15` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:43.130302 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cde753f-dda7-4ad4-89f1-e6a0281cfa66 None admin] RBAC: Authorizing `identity:get_project(project_id=c980d1440d8f49049c8140fffd0d1e15)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:43.131773 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cde753f-dda7-4ad4-89f1-e6a0281cfa66 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:43.133786 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 205/411] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Jul 25 12:01:42 2022] GET /identity/v3/projects/c980d1440d8f49049c8140fffd0d1e15 => generated 302 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:01:43.135885 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:43.142202 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cde753f-dda7-4ad4-89f1-e6a0281cfa66 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:43.197684 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec46c04-5313-45db-964a-53e8b764370a None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:43.263906 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864cde12-4fc0-4596-83a1-dd3cc66ac760 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:43.265197 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864cde12-4fc0-4596-83a1-dd3cc66ac760 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:43.266705 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864cde12-4fc0-4596-83a1-dd3cc66ac760 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:43.268138 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864cde12-4fc0-4596-83a1-dd3cc66ac760 None admin] PATH_INFO: `/v3/groups/ce014e9e4c2c4a64994b15e0fe2ae52a` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:43.269673 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864cde12-4fc0-4596-83a1-dd3cc66ac760 None admin] RBAC: Authorizing `identity:get_group(group_id=ce014e9e4c2c4a64994b15e0fe2ae52a)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:43.272439 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864cde12-4fc0-4596-83a1-dd3cc66ac760 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:43.284733 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 207/412] 10.222.0.42 () {56 vars in 1180 bytes} [Mon Jul 25 12:01:43 2022] GET /identity/v3/groups/ce014e9e4c2c4a64994b15e0fe2ae52a => generated 229 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:01:43.286470 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864cde12-4fc0-4596-83a1-dd3cc66ac760 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:43.324269 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cde753f-dda7-4ad4-89f1-e6a0281cfa66 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:43.378421 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add534f3-589d-4606-bcaa-a9c94392dbdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:43.379318 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add534f3-589d-4606-bcaa-a9c94392dbdc None admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:43.379627 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add534f3-589d-4606-bcaa-a9c94392dbdc None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:43.379911 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add534f3-589d-4606-bcaa-a9c94392dbdc None admin] PATH_INFO: `/v3/projects/c980d1440d8f49049c8140fffd0d1e15/groups/be589db33cdd45c3b8f0684e8ac54c2b/roles/dea3ef72822043f3baf3bcc1be7c318f` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:43.382174 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add534f3-589d-4606-bcaa-a9c94392dbdc None admin] RBAC: Authorizing `identity:create_grant(project_id=c980d1440d8f49049c8140fffd0d1e15, group_id=be589db33cdd45c3b8f0684e8ac54c2b, role_id=dea3ef72822043f3baf3bcc1be7c318f)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:43.383729 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add534f3-589d-4606-bcaa-a9c94392dbdc None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:43.412371 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 206/413] 10.222.0.42 () {58 vars in 1442 bytes} [Mon Jul 25 12:01:43 2022] PUT /identity/v3/projects/c980d1440d8f49049c8140fffd0d1e15/groups/be589db33cdd45c3b8f0684e8ac54c2b/roles/dea3ef72822043f3baf3bcc1be7c318f => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:01:43.417300 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7b6276-2ae7-466c-9ca9-046549b646ff None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:43.418045 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7b6276-2ae7-466c-9ca9-046549b646ff None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:43.419339 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7b6276-2ae7-466c-9ca9-046549b646ff None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:43.421118 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 207/414] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 25 12:01:43 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:01:43.427152 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffe88ac-d67d-4f5c-b7b4-e8c0677f4036 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:43.427830 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffe88ac-d67d-4f5c-b7b4-e8c0677f4036 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:43.428317 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffe88ac-d67d-4f5c-b7b4-e8c0677f4036 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:43.460028 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:43.472113 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864cde12-4fc0-4596-83a1-dd3cc66ac760 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:43.473605 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-fffe88ac-d67d-4f5c-b7b4-e8c0677f4036 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:01:43.476853 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-fffe88ac-d67d-4f5c-b7b4-e8c0677f4036 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:01:43.511742 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:43.551090 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75c7887-dce0-4741-978a-881baf01d9b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:43.552375 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75c7887-dce0-4741-978a-881baf01d9b1 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:43.552945 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75c7887-dce0-4741-978a-881baf01d9b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:43.553422 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75c7887-dce0-4741-978a-881baf01d9b1 None admin] PATH_INFO: `/v3/projects/827585a7cfe8411987f9b81102c1c103` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:43.555610 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75c7887-dce0-4741-978a-881baf01d9b1 None admin] RBAC: Authorizing `identity:get_project(project_id=827585a7cfe8411987f9b81102c1c103)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:43.557318 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75c7887-dce0-4741-978a-881baf01d9b1 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:43.568533 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 208/415] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Jul 25 12:01:43 2022] GET /identity/v3/projects/827585a7cfe8411987f9b81102c1c103 => generated 344 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 12:01:43.571578 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75c7887-dce0-4741-978a-881baf01d9b1 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:43.609624 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 208/416] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 25 12:01:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 12:01:43.640720 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffe88ac-d67d-4f5c-b7b4-e8c0677f4036 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:43.697326 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75c7887-dce0-4741-978a-881baf01d9b1 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:43.737504 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c5b36a-2905-4fe0-ac79-4ea2bd4c3f40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:43.738723 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c5b36a-2905-4fe0-ac79-4ea2bd4c3f40 None admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:43.739214 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c5b36a-2905-4fe0-ac79-4ea2bd4c3f40 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:43.739794 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c5b36a-2905-4fe0-ac79-4ea2bd4c3f40 None admin] PATH_INFO: `/v3/projects/827585a7cfe8411987f9b81102c1c103/groups/ce014e9e4c2c4a64994b15e0fe2ae52a/roles/a7d20885b6d94928a34b3cc5b3a00006` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:43.742102 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c5b36a-2905-4fe0-ac79-4ea2bd4c3f40 None admin] RBAC: Authorizing `identity:create_grant(project_id=827585a7cfe8411987f9b81102c1c103, group_id=ce014e9e4c2c4a64994b15e0fe2ae52a, role_id=a7d20885b6d94928a34b3cc5b3a00006)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:43.743882 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c5b36a-2905-4fe0-ac79-4ea2bd4c3f40 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:43.767878 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffe88ac-d67d-4f5c-b7b4-e8c0677f4036 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:43.794332 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 209/417] 10.222.0.42 () {58 vars in 1442 bytes} [Mon Jul 25 12:01:43 2022] PUT /identity/v3/projects/827585a7cfe8411987f9b81102c1c103/groups/ce014e9e4c2c4a64994b15e0fe2ae52a/roles/a7d20885b6d94928a34b3cc5b3a00006 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:01:43.809925 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad5125e-a12f-43bd-9184-ba6642715644 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:43.810827 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad5125e-a12f-43bd-9184-ba6642715644 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:43.811092 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad5125e-a12f-43bd-9184-ba6642715644 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:43.811377 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad5125e-a12f-43bd-9184-ba6642715644 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:43.814958 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:43.824899 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad5125e-a12f-43bd-9184-ba6642715644 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 12:01:43.834074 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-0ad5125e-a12f-43bd-9184-ba6642715644 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 25 12:01:43.834074 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:01:43.834074 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:01:43.834074 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:01:43.834074 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:01:43.834074 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:01:43.834074 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:01:43.834074 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:01:43.834074 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:01:43.834074 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:01:43.834074 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:01:43.834074 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:01:43.834074 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 12:01:43.834074 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 12:01:43.834074 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 12:01:43.834074 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 12:01:43.835048 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 12:01:43.835048 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 12:01:43.835048 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 12:01:43.835048 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 12:01:43.835048 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:01:43.835048 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:01:43.835048 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:01:43.835048 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:01:43.835048 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:01:43.835048 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:01:43.835048 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:01:43.835048 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jul 25 12:01:43.835048 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:01:43.835048 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:01:43.835048 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:01:43.835048 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:01:43.835834 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:01:43.835834 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:01:43.835834 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:01:43.835834 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:01:43.835834 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 12:01:43.835834 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 12:01:43.835834 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 12:01:43.835834 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 12:01:43.835834 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 12:01:43.835834 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 12:01:43.835834 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 25 12:01:43.835834 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jul 25 12:01:43.835834 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 209/418] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Jul 25 12:01:43 2022] GET /identity/v3/roles/member => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 12:01:43.842772 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c5b36a-2905-4fe0-ac79-4ea2bd4c3f40 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:43.861874 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2b1fd7-9cba-4940-8b97-953920950c46 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:43.862683 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2b1fd7-9cba-4940-8b97-953920950c46 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:43.862683 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2b1fd7-9cba-4940-8b97-953920950c46 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:43.864852 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 210/419] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 25 12:01:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 12:01:43.872892 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341e0bbe-1148-4f19-826d-3c75463f37f0 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:43.873710 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341e0bbe-1148-4f19-826d-3c75463f37f0 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:43.874115 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341e0bbe-1148-4f19-826d-3c75463f37f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:43.910085 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-341e0bbe-1148-4f19-826d-3c75463f37f0 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:01:43.911318 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-341e0bbe-1148-4f19-826d-3c75463f37f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:01:43.960396 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:44.029665 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 211/420] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 25 12:01:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 25 12:01:44.042626 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c5b36a-2905-4fe0-ac79-4ea2bd4c3f40 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:44.053731 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341e0bbe-1148-4f19-826d-3c75463f37f0 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:44.093832 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bfa7c3-02ea-4eec-a5c5-35bc13511510 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:44.094573 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bfa7c3-02ea-4eec-a5c5-35bc13511510 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:44.096451 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bfa7c3-02ea-4eec-a5c5-35bc13511510 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:44.096750 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bfa7c3-02ea-4eec-a5c5-35bc13511510 None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:44.097521 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bfa7c3-02ea-4eec-a5c5-35bc13511510 None admin] RBAC: Adding query filter params (name=member) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:01:44.097840 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bfa7c3-02ea-4eec-a5c5-35bc13511510 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:44.099227 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bfa7c3-02ea-4eec-a5c5-35bc13511510 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:44.108590 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 210/421] 10.222.0.42 () {56 vars in 1101 bytes} [Mon Jul 25 12:01:43 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:01:44.116498 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bfa7c3-02ea-4eec-a5c5-35bc13511510 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:44.226251 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341e0bbe-1148-4f19-826d-3c75463f37f0 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:44.257518 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:44.268486 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb69197-5300-451d-a663-ad1508ce4075 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:44.269652 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb69197-5300-451d-a663-ad1508ce4075 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:44.270637 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb69197-5300-451d-a663-ad1508ce4075 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:44.271133 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb69197-5300-451d-a663-ad1508ce4075 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:44.284421 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb69197-5300-451d-a663-ad1508ce4075 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 12:01:44.294410 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-6cb69197-5300-451d-a663-ad1508ce4075 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 12:01:44.294410 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:01:44.294410 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:01:44.294410 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:01:44.294410 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:01:44.294410 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:01:44.294410 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:01:44.294410 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:01:44.294410 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:01:44.294410 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:01:44.294410 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:01:44.294410 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:01:44.294410 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 12:01:44.294410 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 12:01:44.294410 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 12:01:44.294410 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 12:01:44.295301 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 12:01:44.295301 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 12:01:44.295301 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 12:01:44.295301 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 12:01:44.295301 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:01:44.295301 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:01:44.295301 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:01:44.295301 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:01:44.295301 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:01:44.295301 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:01:44.295301 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:01:44.295301 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jul 25 12:01:44.295301 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:01:44.295301 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:01:44.295301 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:01:44.295301 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:01:44.296175 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:01:44.296175 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:01:44.296175 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:01:44.296175 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:01:44.296175 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 12:01:44.296175 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 12:01:44.296175 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 12:01:44.296175 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 12:01:44.296175 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 12:01:44.296175 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 12:01:44.296175 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 12:01:44.296175 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jul 25 12:01:44.299651 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 212/422] 10.222.0.42 () {56 vars in 1114 bytes} [Mon Jul 25 12:01:44 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 12:01:44.312533 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb69197-5300-451d-a663-ad1508ce4075 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:44.322231 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bfa7c3-02ea-4eec-a5c5-35bc13511510 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:44.332417 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:44.383486 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381993f5-862b-4f3f-a9f1-897d96ba126b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:44.384178 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381993f5-862b-4f3f-a9f1-897d96ba126b None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:44.384468 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381993f5-862b-4f3f-a9f1-897d96ba126b None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:44.384758 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381993f5-862b-4f3f-a9f1-897d96ba126b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:44.485935 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb69197-5300-451d-a663-ad1508ce4075 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:44.521952 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f5b4ad-8b6c-4381-a4b1-908a2b6fcd3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:44.522634 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f5b4ad-8b6c-4381-a4b1-908a2b6fcd3c None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:44.522921 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f5b4ad-8b6c-4381-a4b1-908a2b6fcd3c None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:44.523203 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f5b4ad-8b6c-4381-a4b1-908a2b6fcd3c None admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:44.523986 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f5b4ad-8b6c-4381-a4b1-908a2b6fcd3c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:01:44.524302 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f5b4ad-8b6c-4381-a4b1-908a2b6fcd3c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:44.525817 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f5b4ad-8b6c-4381-a4b1-908a2b6fcd3c None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:44.535788 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 213/423] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Jul 25 12:01:44 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:01:44.543738 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f5b4ad-8b6c-4381-a4b1-908a2b6fcd3c None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:44.552473 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381993f5-862b-4f3f-a9f1-897d96ba126b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:44.553945 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381993f5-862b-4f3f-a9f1-897d96ba126b None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:44.658025 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 211/424] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Jul 25 12:01:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 541 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 12:01:44.665296 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381993f5-862b-4f3f-a9f1-897d96ba126b None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:44.673541 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:44.702569 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:44.774336 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381993f5-862b-4f3f-a9f1-897d96ba126b None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:44.810414 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6539f0a5-6704-4293-8df1-cb979d04436a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:44.811286 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6539f0a5-6704-4293-8df1-cb979d04436a None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:44.811601 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6539f0a5-6704-4293-8df1-cb979d04436a None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:44.811899 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6539f0a5-6704-4293-8df1-cb979d04436a None admin] PATH_INFO: `/v3/projects/23c9d26ac78646c79204b87bd5332e2c` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:44.812888 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f5b4ad-8b6c-4381-a4b1-908a2b6fcd3c None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:44.813104 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6539f0a5-6704-4293-8df1-cb979d04436a None admin] RBAC: Authorizing `identity:get_project(project_id=23c9d26ac78646c79204b87bd5332e2c)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:44.815451 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6539f0a5-6704-4293-8df1-cb979d04436a None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:44.817582 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 212/425] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Jul 25 12:01:44 2022] GET /identity/v3/projects/23c9d26ac78646c79204b87bd5332e2c => generated 298 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:01:44.828628 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6539f0a5-6704-4293-8df1-cb979d04436a None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:44.848443 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0921d7-4824-4e1d-b37e-a74303a7ccb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:44.849105 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0921d7-4824-4e1d-b37e-a74303a7ccb9 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:44.849397 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0921d7-4824-4e1d-b37e-a74303a7ccb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:44.849681 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0921d7-4824-4e1d-b37e-a74303a7ccb9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:44.925991 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0921d7-4824-4e1d-b37e-a74303a7ccb9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:44.927478 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0921d7-4824-4e1d-b37e-a74303a7ccb9 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:44.952635 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:44.983823 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:44.994781 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6539f0a5-6704-4293-8df1-cb979d04436a None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:45.039813 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086c3c10-bccc-4ff7-a9e1-541c07e34860 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:45.040276 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 214/426] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Jul 25 12:01:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 497 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 25 12:01:45.041605 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086c3c10-bccc-4ff7-a9e1-541c07e34860 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:45.041990 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bf7e0e-fc4d-4386-8d1c-ccddc587133a None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:45.042222 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086c3c10-bccc-4ff7-a9e1-541c07e34860 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:45.042468 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bf7e0e-fc4d-4386-8d1c-ccddc587133a None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:45.042790 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086c3c10-bccc-4ff7-a9e1-541c07e34860 None admin] PATH_INFO: `/v3/groups/be589db33cdd45c3b8f0684e8ac54c2b` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:45.042898 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bf7e0e-fc4d-4386-8d1c-ccddc587133a None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:45.044254 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086c3c10-bccc-4ff7-a9e1-541c07e34860 None admin] RBAC: Authorizing `identity:get_group(group_id=be589db33cdd45c3b8f0684e8ac54c2b)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:45.044612 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 215/427] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 25 12:01:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:01:45.046084 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086c3c10-bccc-4ff7-a9e1-541c07e34860 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:45.051593 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 213/428] 10.222.0.42 () {56 vars in 1180 bytes} [Mon Jul 25 12:01:44 2022] GET /identity/v3/groups/be589db33cdd45c3b8f0684e8ac54c2b => generated 226 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:01:45.053913 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086c3c10-bccc-4ff7-a9e1-541c07e34860 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:45.054042 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18c2862-cef7-4f7a-866c-d6be4279a675 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:45.054319 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18c2862-cef7-4f7a-866c-d6be4279a675 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:45.054602 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18c2862-cef7-4f7a-866c-d6be4279a675 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:45.074067 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-c18c2862-cef7-4f7a-866c-d6be4279a675 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:01:45.075176 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-c18c2862-cef7-4f7a-866c-d6be4279a675 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:01:45.155424 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 216/429] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 25 12:01:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 12:01:45.156409 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18c2862-cef7-4f7a-866c-d6be4279a675 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:45.168584 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086c3c10-bccc-4ff7-a9e1-541c07e34860 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:45.208534 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec98fe3f-c7d8-4460-908d-c661ee9b692a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:45.209537 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec98fe3f-c7d8-4460-908d-c661ee9b692a None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:45.209832 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec98fe3f-c7d8-4460-908d-c661ee9b692a None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:45.210114 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec98fe3f-c7d8-4460-908d-c661ee9b692a None admin] PATH_INFO: `/v3/projects/23c9d26ac78646c79204b87bd5332e2c` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:45.211527 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec98fe3f-c7d8-4460-908d-c661ee9b692a None admin] RBAC: Authorizing `identity:get_project(project_id=23c9d26ac78646c79204b87bd5332e2c)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:45.213022 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec98fe3f-c7d8-4460-908d-c661ee9b692a None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:45.215287 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 214/430] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Jul 25 12:01:45 2022] GET /identity/v3/projects/23c9d26ac78646c79204b87bd5332e2c => generated 298 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:01:45.216154 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec98fe3f-c7d8-4460-908d-c661ee9b692a None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:45.229508 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:45.245710 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:45.269208 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18c2862-cef7-4f7a-866c-d6be4279a675 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:45.307662 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbb237b-41a2-4f84-9e42-4161f21726c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:45.308323 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbb237b-41a2-4f84-9e42-4161f21726c5 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:45.308695 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbb237b-41a2-4f84-9e42-4161f21726c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:45.308976 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbb237b-41a2-4f84-9e42-4161f21726c5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:45.309875 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbb237b-41a2-4f84-9e42-4161f21726c5 None admin] RBAC: Adding query filter params (group.id=be589db33cdd45c3b8f0684e8ac54c2b, role.id=472313ae69174578824df0bed6390744, scope.project.id=23c9d26ac78646c79204b87bd5332e2c) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:01:45.310194 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbb237b-41a2-4f84-9e42-4161f21726c5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:45.311729 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbb237b-41a2-4f84-9e42-4161f21726c5 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:45.318648 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 217/431] 10.222.0.42 () {56 vars in 1376 bytes} [Mon Jul 25 12:01:45 2022] GET /identity/v3/role_assignments?group.id=be589db33cdd45c3b8f0684e8ac54c2b&scope.project.id=23c9d26ac78646c79204b87bd5332e2c&role.id=472313ae69174578824df0bed6390744 => generated 611 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:01:45.319272 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbb237b-41a2-4f84-9e42-4161f21726c5 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:45.341749 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec98fe3f-c7d8-4460-908d-c661ee9b692a None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:45.387204 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977882d1-e95c-42a1-883a-951c1cfe5dd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:45.388462 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977882d1-e95c-42a1-883a-951c1cfe5dd3 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:45.388768 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977882d1-e95c-42a1-883a-951c1cfe5dd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:45.389053 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977882d1-e95c-42a1-883a-951c1cfe5dd3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:45.401768 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977882d1-e95c-42a1-883a-951c1cfe5dd3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 12:01:45.411068 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-977882d1-e95c-42a1-883a-951c1cfe5dd3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 25 12:01:45.411068 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:01:45.411068 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:01:45.411068 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:01:45.411068 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:01:45.411068 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:01:45.411068 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:01:45.411068 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:01:45.411068 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:01:45.411068 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:01:45.411068 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:01:45.411068 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:01:45.411068 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 12:01:45.411068 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 12:01:45.411068 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 12:01:45.411068 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 12:01:45.412064 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 12:01:45.412064 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 12:01:45.412064 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 12:01:45.412064 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 12:01:45.412064 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:01:45.412064 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:01:45.412064 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:01:45.412064 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:01:45.412064 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:01:45.412064 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:01:45.412064 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:01:45.412064 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 25 12:01:45.412064 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:01:45.412064 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:01:45.412064 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:01:45.412064 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:01:45.412948 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:01:45.412948 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:01:45.412948 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:01:45.412948 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:01:45.412948 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 12:01:45.412948 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 12:01:45.412948 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 12:01:45.412948 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 12:01:45.412948 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 12:01:45.412948 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 12:01:45.412948 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 25 12:01:45.412948 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 25 12:01:45.413547 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 215/432] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Jul 25 12:01:45 2022] GET /identity/v3/roles/member => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:01:45.419965 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977882d1-e95c-42a1-883a-951c1cfe5dd3 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:45.521448 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:45.556031 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbb237b-41a2-4f84-9e42-4161f21726c5 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:45.559827 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977882d1-e95c-42a1-883a-951c1cfe5dd3 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:45.596359 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fa5828-7c60-441e-81f9-369012927912 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:45.597201 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fa5828-7c60-441e-81f9-369012927912 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:45.597496 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fa5828-7c60-441e-81f9-369012927912 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:45.597778 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fa5828-7c60-441e-81f9-369012927912 None admin] PATH_INFO: `/v3/groups/be589db33cdd45c3b8f0684e8ac54c2b` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:45.599164 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52a3649-74c0-486d-bc56-79734d5cf8b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:45.599624 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fa5828-7c60-441e-81f9-369012927912 None admin] RBAC: Authorizing `identity:get_group(group_id=be589db33cdd45c3b8f0684e8ac54c2b)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:45.599871 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52a3649-74c0-486d-bc56-79734d5cf8b5 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:45.600163 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52a3649-74c0-486d-bc56-79734d5cf8b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:45.600446 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52a3649-74c0-486d-bc56-79734d5cf8b5 None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:45.601063 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fa5828-7c60-441e-81f9-369012927912 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:45.601233 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52a3649-74c0-486d-bc56-79734d5cf8b5 None admin] RBAC: Adding query filter params (name=member) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:01:45.601499 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52a3649-74c0-486d-bc56-79734d5cf8b5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:45.602858 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52a3649-74c0-486d-bc56-79734d5cf8b5 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:45.603032 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 218/433] 10.222.0.42 () {56 vars in 1180 bytes} [Mon Jul 25 12:01:45 2022] GET /identity/v3/groups/be589db33cdd45c3b8f0684e8ac54c2b => generated 226 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:01:45.608625 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fa5828-7c60-441e-81f9-369012927912 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:45.612082 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 216/434] 10.222.0.42 () {56 vars in 1101 bytes} [Mon Jul 25 12:01:45 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:01:45.620980 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52a3649-74c0-486d-bc56-79734d5cf8b5 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:45.626682 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:45.732711 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52a3649-74c0-486d-bc56-79734d5cf8b5 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:45.738125 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fa5828-7c60-441e-81f9-369012927912 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:45.764770 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:45.770782 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848477d6-48a2-4df7-a4a8-e504f8ff8223 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:45.771730 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848477d6-48a2-4df7-a4a8-e504f8ff8223 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:45.772290 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848477d6-48a2-4df7-a4a8-e504f8ff8223 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:45.772775 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848477d6-48a2-4df7-a4a8-e504f8ff8223 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:45.776697 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8b51d7-cea7-43a1-ad0a-3a0f303af74a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:45.777368 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8b51d7-cea7-43a1-ad0a-3a0f303af74a None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:45.777663 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8b51d7-cea7-43a1-ad0a-3a0f303af74a None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:45.777950 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8b51d7-cea7-43a1-ad0a-3a0f303af74a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:45.779579 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8b51d7-cea7-43a1-ad0a-3a0f303af74a None admin] RBAC: Adding query filter params (group.id=be589db33cdd45c3b8f0684e8ac54c2b, role.id=dea3ef72822043f3baf3bcc1be7c318f, scope.project.id=23c9d26ac78646c79204b87bd5332e2c) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:01:45.779579 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8b51d7-cea7-43a1-ad0a-3a0f303af74a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:45.780738 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8b51d7-cea7-43a1-ad0a-3a0f303af74a None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:45.788146 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 219/435] 10.222.0.42 () {56 vars in 1376 bytes} [Mon Jul 25 12:01:45 2022] GET /identity/v3/role_assignments?group.id=be589db33cdd45c3b8f0684e8ac54c2b&scope.project.id=23c9d26ac78646c79204b87bd5332e2c&role.id=dea3ef72822043f3baf3bcc1be7c318f => generated 611 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:01:45.857726 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848477d6-48a2-4df7-a4a8-e504f8ff8223 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:45.859538 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848477d6-48a2-4df7-a4a8-e504f8ff8223 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:45.890373 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc7cdb1-0923-48da-831c-311d711078bf None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:45.890686 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc7cdb1-0923-48da-831c-311d711078bf None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:45.890935 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc7cdb1-0923-48da-831c-311d711078bf None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:45.897014 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 220/436] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 25 12:01:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:01:45.898919 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e841d8-e51f-471c-b930-b6e11a07ebbc None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:45.899278 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e841d8-e51f-471c-b930-b6e11a07ebbc None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:45.899704 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e841d8-e51f-471c-b930-b6e11a07ebbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:45.923250 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-e4e841d8-e51f-471c-b930-b6e11a07ebbc None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:01:45.924347 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-e4e841d8-e51f-471c-b930-b6e11a07ebbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:01:45.978467 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 217/437] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Jul 25 12:01:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 12:01:45.984291 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848477d6-48a2-4df7-a4a8-e504f8ff8223 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:46.018866 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 221/438] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 25 12:01:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 25 12:01:46.022233 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:46.044299 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e841d8-e51f-471c-b930-b6e11a07ebbc None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:46.048741 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:46.116579 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848477d6-48a2-4df7-a4a8-e504f8ff8223 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:46.160267 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b459d817-f2f1-4468-8d8d-5259eacff67d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:46.161202 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b459d817-f2f1-4468-8d8d-5259eacff67d None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:46.161499 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b459d817-f2f1-4468-8d8d-5259eacff67d None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:46.161779 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b459d817-f2f1-4468-8d8d-5259eacff67d None admin] PATH_INFO: `/v3/projects/c980d1440d8f49049c8140fffd0d1e15` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:46.162976 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b459d817-f2f1-4468-8d8d-5259eacff67d None admin] RBAC: Authorizing `identity:get_project(project_id=c980d1440d8f49049c8140fffd0d1e15)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:46.164558 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e841d8-e51f-471c-b930-b6e11a07ebbc None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:46.165939 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b459d817-f2f1-4468-8d8d-5259eacff67d None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:46.168902 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 218/439] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Jul 25 12:01:45 2022] GET /identity/v3/projects/c980d1440d8f49049c8140fffd0d1e15 => generated 302 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 12:01:46.173583 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b459d817-f2f1-4468-8d8d-5259eacff67d None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:46.212544 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ace74e-653c-4bd9-86d2-96dc0ba5428a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:46.213524 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ace74e-653c-4bd9-86d2-96dc0ba5428a None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:46.213820 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ace74e-653c-4bd9-86d2-96dc0ba5428a None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:46.214136 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ace74e-653c-4bd9-86d2-96dc0ba5428a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:46.226559 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ace74e-653c-4bd9-86d2-96dc0ba5428a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 12:01:46.235236 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-b5ace74e-653c-4bd9-86d2-96dc0ba5428a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 12:01:46.235236 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:01:46.235236 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:01:46.235236 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:01:46.235236 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:01:46.235236 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:01:46.235236 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:01:46.235236 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:01:46.235236 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:01:46.235236 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:01:46.235236 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:01:46.235236 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:01:46.235236 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 12:01:46.235236 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 12:01:46.235236 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 12:01:46.235236 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 12:01:46.236769 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 12:01:46.236769 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 12:01:46.236769 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 12:01:46.236769 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 12:01:46.236769 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:01:46.236769 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:01:46.236769 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:01:46.236769 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:01:46.236769 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:01:46.236769 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:01:46.236769 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:01:46.236769 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jul 25 12:01:46.236769 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:01:46.236769 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:01:46.236769 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:01:46.236769 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:01:46.237564 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:01:46.237564 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:01:46.237564 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:01:46.237564 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:01:46.237564 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 12:01:46.237564 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 12:01:46.237564 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 12:01:46.237564 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 12:01:46.237564 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 12:01:46.237564 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 12:01:46.237564 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 12:01:46.237564 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jul 25 12:01:46.237564 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 222/440] 10.222.0.42 () {56 vars in 1114 bytes} [Mon Jul 25 12:01:46 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 12:01:46.238791 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fe8009-2ea6-466d-ba90-40ed69577847 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:46.238897 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fe8009-2ea6-466d-ba90-40ed69577847 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:46.239141 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fe8009-2ea6-466d-ba90-40ed69577847 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:46.240443 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 223/441] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 25 12:01:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:01:46.245142 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fe8009-2ea6-466d-ba90-40ed69577847 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:46.293731 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:46.296866 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b459d817-f2f1-4468-8d8d-5259eacff67d None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:46.324851 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:46.339523 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfda8a16-6f81-4576-ad6f-c65a32c31282 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:46.340197 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfda8a16-6f81-4576-ad6f-c65a32c31282 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:46.340484 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfda8a16-6f81-4576-ad6f-c65a32c31282 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:46.340769 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfda8a16-6f81-4576-ad6f-c65a32c31282 None admin] PATH_INFO: `/v3/groups/be589db33cdd45c3b8f0684e8ac54c2b` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:46.341974 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfda8a16-6f81-4576-ad6f-c65a32c31282 None admin] RBAC: Authorizing `identity:get_group(group_id=be589db33cdd45c3b8f0684e8ac54c2b)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:46.343639 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfda8a16-6f81-4576-ad6f-c65a32c31282 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:46.346949 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 219/442] 10.222.0.42 () {56 vars in 1180 bytes} [Mon Jul 25 12:01:46 2022] GET /identity/v3/groups/be589db33cdd45c3b8f0684e8ac54c2b => generated 226 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:01:46.348188 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b6953d-dcf2-41d3-841f-14f7fd549667 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:46.348188 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b6953d-dcf2-41d3-841f-14f7fd549667 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:46.348468 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b6953d-dcf2-41d3-841f-14f7fd549667 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:46.368337 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-63b6953d-dcf2-41d3-841f-14f7fd549667 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:01:46.369465 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-63b6953d-dcf2-41d3-841f-14f7fd549667 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:01:46.370969 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fe8009-2ea6-466d-ba90-40ed69577847 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:46.403003 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0524c3-5acb-491c-a215-df0cd5137222 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:46.403694 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0524c3-5acb-491c-a215-df0cd5137222 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:46.403982 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0524c3-5acb-491c-a215-df0cd5137222 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:46.404261 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0524c3-5acb-491c-a215-df0cd5137222 None admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:46.405001 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0524c3-5acb-491c-a215-df0cd5137222 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:01:46.405312 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0524c3-5acb-491c-a215-df0cd5137222 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:46.406665 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0524c3-5acb-491c-a215-df0cd5137222 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:46.414980 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 224/443] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Jul 25 12:01:46 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:01:46.416119 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0524c3-5acb-491c-a215-df0cd5137222 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:46.439393 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 220/444] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 25 12:01:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 12:01:46.440532 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b6953d-dcf2-41d3-841f-14f7fd549667 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:46.520068 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0524c3-5acb-491c-a215-df0cd5137222 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:46.538025 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:46.541801 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b6953d-dcf2-41d3-841f-14f7fd549667 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:46.545337 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:46.554523 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3692eaf4-9a41-4515-888b-835fc7282ced None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:46.555172 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3692eaf4-9a41-4515-888b-835fc7282ced None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:46.555456 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3692eaf4-9a41-4515-888b-835fc7282ced None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:46.555769 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3692eaf4-9a41-4515-888b-835fc7282ced None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:46.556598 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3692eaf4-9a41-4515-888b-835fc7282ced None admin] RBAC: Adding query filter params (group.id=be589db33cdd45c3b8f0684e8ac54c2b, role.id=472313ae69174578824df0bed6390744, scope.project.id=c980d1440d8f49049c8140fffd0d1e15) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:01:46.556912 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3692eaf4-9a41-4515-888b-835fc7282ced None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:46.558347 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3692eaf4-9a41-4515-888b-835fc7282ced None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:46.564550 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 225/445] 10.222.0.42 () {56 vars in 1376 bytes} [Mon Jul 25 12:01:46 2022] GET /identity/v3/role_assignments?group.id=be589db33cdd45c3b8f0684e8ac54c2b&scope.project.id=c980d1440d8f49049c8140fffd0d1e15&role.id=472313ae69174578824df0bed6390744 => generated 611 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:01:46.565491 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3692eaf4-9a41-4515-888b-835fc7282ced None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:46.588835 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e97b3a-7aa3-4d17-b033-f017f13e3bff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:46.589516 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e97b3a-7aa3-4d17-b033-f017f13e3bff None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:46.589810 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e97b3a-7aa3-4d17-b033-f017f13e3bff None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:46.590094 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e97b3a-7aa3-4d17-b033-f017f13e3bff None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:46.670086 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e97b3a-7aa3-4d17-b033-f017f13e3bff None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:46.671742 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e97b3a-7aa3-4d17-b033-f017f13e3bff None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:46.678483 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3692eaf4-9a41-4515-888b-835fc7282ced None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:46.714267 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7ab365-72bc-4561-b52a-b7823997a550 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:46.715069 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7ab365-72bc-4561-b52a-b7823997a550 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:46.715353 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7ab365-72bc-4561-b52a-b7823997a550 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:46.715659 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7ab365-72bc-4561-b52a-b7823997a550 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:46.728955 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7ab365-72bc-4561-b52a-b7823997a550 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 12:01:46.735977 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-ed7ab365-72bc-4561-b52a-b7823997a550 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 12:01:46.735977 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:01:46.735977 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:01:46.735977 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:01:46.735977 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:01:46.735977 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:01:46.735977 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:01:46.735977 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:01:46.735977 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:01:46.735977 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:01:46.735977 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:01:46.735977 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:01:46.735977 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 12:01:46.735977 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 12:01:46.735977 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 12:01:46.735977 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 12:01:46.737055 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 12:01:46.737055 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 12:01:46.737055 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 12:01:46.737055 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 12:01:46.737055 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:01:46.737055 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:01:46.737055 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:01:46.737055 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:01:46.737055 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:01:46.737055 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:01:46.737055 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:01:46.737055 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jul 25 12:01:46.737055 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:01:46.737055 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:01:46.737055 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:01:46.737055 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:01:46.737853 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:01:46.737853 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:01:46.737853 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:01:46.737853 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:01:46.737853 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 12:01:46.737853 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 12:01:46.737853 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 12:01:46.737853 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 12:01:46.737853 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 12:01:46.737853 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 12:01:46.737853 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 12:01:46.737853 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jul 25 12:01:46.737853 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 226/446] 10.222.0.42 () {56 vars in 1096 bytes} [Mon Jul 25 12:01:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:01:46.743229 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7ab365-72bc-4561-b52a-b7823997a550 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:46.785602 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:46.793577 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 221/447] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Jul 25 12:01:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 12:01:46.799014 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e97b3a-7aa3-4d17-b033-f017f13e3bff None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:46.814121 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:46.873328 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7ab365-72bc-4561-b52a-b7823997a550 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:46.914646 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8c7f4c-80e7-4dd0-b826-7d3e1a5865a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:46.915344 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8c7f4c-80e7-4dd0-b826-7d3e1a5865a5 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:46.915980 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8c7f4c-80e7-4dd0-b826-7d3e1a5865a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:46.916262 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8c7f4c-80e7-4dd0-b826-7d3e1a5865a5 None admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:46.917010 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8c7f4c-80e7-4dd0-b826-7d3e1a5865a5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:01:46.917325 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8c7f4c-80e7-4dd0-b826-7d3e1a5865a5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:46.918701 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8c7f4c-80e7-4dd0-b826-7d3e1a5865a5 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:46.920755 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e97b3a-7aa3-4d17-b033-f017f13e3bff None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:46.927345 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 227/448] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Jul 25 12:01:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 12:01:46.932964 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8c7f4c-80e7-4dd0-b826-7d3e1a5865a5 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:46.955876 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb41ba2b-9ba7-40ba-8d69-7e189e32d744 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:46.956726 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb41ba2b-9ba7-40ba-8d69-7e189e32d744 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:46.957009 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb41ba2b-9ba7-40ba-8d69-7e189e32d744 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:46.957285 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb41ba2b-9ba7-40ba-8d69-7e189e32d744 None admin] PATH_INFO: `/v3/projects/c980d1440d8f49049c8140fffd0d1e15` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:46.958457 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb41ba2b-9ba7-40ba-8d69-7e189e32d744 None admin] RBAC: Authorizing `identity:get_project(project_id=c980d1440d8f49049c8140fffd0d1e15)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:46.959961 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb41ba2b-9ba7-40ba-8d69-7e189e32d744 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:46.961823 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 222/449] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Jul 25 12:01:46 2022] GET /identity/v3/projects/c980d1440d8f49049c8140fffd0d1e15 => generated 302 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 12:01:46.968987 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb41ba2b-9ba7-40ba-8d69-7e189e32d744 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:47.031864 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8c7f4c-80e7-4dd0-b826-7d3e1a5865a5 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:47.032956 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:47.045175 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:47.067658 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb41ba2b-9ba7-40ba-8d69-7e189e32d744 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:47.067822 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7770e4c5-4522-4a07-8dc5-2755ff164515 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:47.068471 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7770e4c5-4522-4a07-8dc5-2755ff164515 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:47.068761 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7770e4c5-4522-4a07-8dc5-2755ff164515 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:47.069047 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7770e4c5-4522-4a07-8dc5-2755ff164515 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:47.102065 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b732dfb1-06c7-40ce-9020-22f5f608a513 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:47.102915 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b732dfb1-06c7-40ce-9020-22f5f608a513 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:47.103262 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b732dfb1-06c7-40ce-9020-22f5f608a513 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:47.103584 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b732dfb1-06c7-40ce-9020-22f5f608a513 None admin] PATH_INFO: `/v3/groups/be589db33cdd45c3b8f0684e8ac54c2b` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:47.104742 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b732dfb1-06c7-40ce-9020-22f5f608a513 None admin] RBAC: Authorizing `identity:get_group(group_id=be589db33cdd45c3b8f0684e8ac54c2b)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:47.106189 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b732dfb1-06c7-40ce-9020-22f5f608a513 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:47.107844 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 223/450] 10.222.0.42 () {56 vars in 1180 bytes} [Mon Jul 25 12:01:46 2022] GET /identity/v3/groups/be589db33cdd45c3b8f0684e8ac54c2b => generated 226 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 12:01:47.114276 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b732dfb1-06c7-40ce-9020-22f5f608a513 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:47.140792 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7770e4c5-4522-4a07-8dc5-2755ff164515 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:47.142277 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7770e4c5-4522-4a07-8dc5-2755ff164515 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:47.210060 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b732dfb1-06c7-40ce-9020-22f5f608a513 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:47.239240 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 228/451] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Jul 25 12:01:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 25 12:01:47.243438 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b382f15a-e478-483b-a328-a741c6d4b017 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:47.244380 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b382f15a-e478-483b-a328-a741c6d4b017 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:47.244670 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b382f15a-e478-483b-a328-a741c6d4b017 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:47.244871 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7770e4c5-4522-4a07-8dc5-2755ff164515 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:47.244961 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b382f15a-e478-483b-a328-a741c6d4b017 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:47.245778 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b382f15a-e478-483b-a328-a741c6d4b017 None admin] RBAC: Adding query filter params (group.id=be589db33cdd45c3b8f0684e8ac54c2b, role.id=dea3ef72822043f3baf3bcc1be7c318f, scope.project.id=c980d1440d8f49049c8140fffd0d1e15) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:01:47.246093 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b382f15a-e478-483b-a328-a741c6d4b017 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:47.247527 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b382f15a-e478-483b-a328-a741c6d4b017 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:47.249381 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:47.255580 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 224/452] 10.222.0.42 () {56 vars in 1376 bytes} [Mon Jul 25 12:01:47 2022] GET /identity/v3/role_assignments?group.id=be589db33cdd45c3b8f0684e8ac54c2b&scope.project.id=c980d1440d8f49049c8140fffd0d1e15&role.id=dea3ef72822043f3baf3bcc1be7c318f => generated 611 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 12:01:47.258145 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:47.349079 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7770e4c5-4522-4a07-8dc5-2755ff164515 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:47.383364 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150aa771-bbf6-4a11-ba7e-6848eb127bde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:47.384263 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150aa771-bbf6-4a11-ba7e-6848eb127bde None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:47.384549 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150aa771-bbf6-4a11-ba7e-6848eb127bde None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:47.384827 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150aa771-bbf6-4a11-ba7e-6848eb127bde None admin] PATH_INFO: `/v3/projects/827585a7cfe8411987f9b81102c1c103` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:47.386058 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150aa771-bbf6-4a11-ba7e-6848eb127bde None admin] RBAC: Authorizing `identity:get_project(project_id=827585a7cfe8411987f9b81102c1c103)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:47.387521 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150aa771-bbf6-4a11-ba7e-6848eb127bde None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:47.389258 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 229/453] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Jul 25 12:01:47 2022] GET /identity/v3/projects/827585a7cfe8411987f9b81102c1c103 => generated 344 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 12:01:47.395456 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b382f15a-e478-483b-a328-a741c6d4b017 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:47.499001 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b382f15a-e478-483b-a328-a741c6d4b017 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:47.532560 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6143e549-4c24-4489-802b-178940ca94b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:47.533424 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6143e549-4c24-4489-802b-178940ca94b8 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:47.533708 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6143e549-4c24-4489-802b-178940ca94b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:47.533988 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6143e549-4c24-4489-802b-178940ca94b8 None admin] PATH_INFO: `/v3/groups/ce014e9e4c2c4a64994b15e0fe2ae52a` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:47.535164 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6143e549-4c24-4489-802b-178940ca94b8 None admin] RBAC: Authorizing `identity:get_group(group_id=ce014e9e4c2c4a64994b15e0fe2ae52a)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:47.536654 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6143e549-4c24-4489-802b-178940ca94b8 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:47.538307 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 225/454] 10.222.0.42 () {56 vars in 1180 bytes} [Mon Jul 25 12:01:47 2022] GET /identity/v3/groups/ce014e9e4c2c4a64994b15e0fe2ae52a => generated 229 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:01:47.544464 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150aa771-bbf6-4a11-ba7e-6848eb127bde None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:47.626646 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:47.645499 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150aa771-bbf6-4a11-ba7e-6848eb127bde None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:47.678146 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1886aa0-da0e-4b9f-ad67-ce7b586bed26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:47.678792 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1886aa0-da0e-4b9f-ad67-ce7b586bed26 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:47.679076 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1886aa0-da0e-4b9f-ad67-ce7b586bed26 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:47.679353 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1886aa0-da0e-4b9f-ad67-ce7b586bed26 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:47.680208 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1886aa0-da0e-4b9f-ad67-ce7b586bed26 None admin] RBAC: Adding query filter params (group.id=ce014e9e4c2c4a64994b15e0fe2ae52a, role.id=a7d20885b6d94928a34b3cc5b3a00006, scope.project.id=827585a7cfe8411987f9b81102c1c103) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:01:47.680519 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1886aa0-da0e-4b9f-ad67-ce7b586bed26 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:47.681944 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1886aa0-da0e-4b9f-ad67-ce7b586bed26 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:47.687579 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 230/455] 10.222.0.42 () {56 vars in 1376 bytes} [Mon Jul 25 12:01:47 2022] GET /identity/v3/role_assignments?group.id=ce014e9e4c2c4a64994b15e0fe2ae52a&scope.project.id=827585a7cfe8411987f9b81102c1c103&role.id=a7d20885b6d94928a34b3cc5b3a00006 => generated 611 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:01:50.180370 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272aaebd-f9e9-4a7d-b4dd-9fdce7f70c4f None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:50.180890 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272aaebd-f9e9-4a7d-b4dd-9fdce7f70c4f None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:50.180965 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272aaebd-f9e9-4a7d-b4dd-9fdce7f70c4f None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:50.182641 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 226/456] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 25 12:01:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:01:50.187811 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16275910-3c7c-42bb-a080-987bfa7282c7 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:50.188240 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16275910-3c7c-42bb-a080-987bfa7282c7 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:50.188409 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16275910-3c7c-42bb-a080-987bfa7282c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:50.214612 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cd63cf-c6f3-4eb1-8e17-6a99cd8cec06 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:50.215230 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cd63cf-c6f3-4eb1-8e17-6a99cd8cec06 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:50.215774 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cd63cf-c6f3-4eb1-8e17-6a99cd8cec06 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:50.217704 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 227/457] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 25 12:01:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 12:01:50.218656 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-16275910-3c7c-42bb-a080-987bfa7282c7 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:01:50.219971 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-16275910-3c7c-42bb-a080-987bfa7282c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:01:50.223532 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08686f97-1b9d-46cb-8960-e938a8872eeb None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:50.224121 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08686f97-1b9d-46cb-8960-e938a8872eeb None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:50.224605 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08686f97-1b9d-46cb-8960-e938a8872eeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:50.255647 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-08686f97-1b9d-46cb-8960-e938a8872eeb None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:01:50.256870 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-08686f97-1b9d-46cb-8960-e938a8872eeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:01:50.329626 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 231/458] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 25 12:01:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 25 12:01:50.357927 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:50.369524 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 228/459] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 25 12:01:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 25 12:01:50.375027 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16275910-3c7c-42bb-a080-987bfa7282c7 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:50.394257 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08686f97-1b9d-46cb-8960-e938a8872eeb None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:50.505959 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:50.554759 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08686f97-1b9d-46cb-8960-e938a8872eeb None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:50.568018 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16275910-3c7c-42bb-a080-987bfa7282c7 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:50.596951 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d770c208-b8e5-4154-899c-d8f4418ca520 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:50.597844 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d770c208-b8e5-4154-899c-d8f4418ca520 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:50.598211 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d770c208-b8e5-4154-899c-d8f4418ca520 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:50.598556 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d770c208-b8e5-4154-899c-d8f4418ca520 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:50.609362 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d770c208-b8e5-4154-899c-d8f4418ca520 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:50.611152 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d770c208-b8e5-4154-899c-d8f4418ca520 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:50.625116 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcf3895-b5e5-473e-aa2c-2fbb76a680ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:50.626167 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcf3895-b5e5-473e-aa2c-2fbb76a680ab None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:50.626633 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcf3895-b5e5-473e-aa2c-2fbb76a680ab None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:50.627109 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcf3895-b5e5-473e-aa2c-2fbb76a680ab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:50.632385 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-d770c208-b8e5-4154-899c-d8f4418ca520 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:01:50.632385 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:01:50.632385 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:01:50.632385 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:01:50.632385 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:01:50.632385 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:01:50.632385 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:01:50.632385 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:01:50.632385 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:01:50.632385 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:01:50.632385 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:01:50.632385 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:01:50.632385 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 12:01:50.632385 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 12:01:50.632385 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 12:01:50.633241 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 12:01:50.633241 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:01:50.633241 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:01:50.633241 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:01:50.633241 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:01:50.633241 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:01:50.633241 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:01:50.633241 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:01:50.633241 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 25 12:01:50.633241 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:01:50.633241 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:01:50.633241 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:01:50.633241 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:01:50.633241 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:01:50.633241 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:01:50.633241 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:01:50.633241 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:01:50.634090 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 12:01:50.634090 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 12:01:50.634090 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:01:50.634090 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 25 12:01:50.634090 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 229/460] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jul 25 12:01:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 12:01:50.640091 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533a9489-c91c-4d20-9a3d-dc90313f1ce8 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:50.640357 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533a9489-c91c-4d20-9a3d-dc90313f1ce8 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:50.640647 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533a9489-c91c-4d20-9a3d-dc90313f1ce8 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:50.641439 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bcf3895-b5e5-473e-aa2c-2fbb76a680ab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:50.641965 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 230/461] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 25 12:01:50 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:01:50.642673 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533a9489-c91c-4d20-9a3d-dc90313f1ce8 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:50.644186 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bcf3895-b5e5-473e-aa2c-2fbb76a680ab None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:50.667161 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-0bcf3895-b5e5-473e-aa2c-2fbb76a680ab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:01:50.667161 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:01:50.667161 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:01:50.667161 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:01:50.667161 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:01:50.667161 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:01:50.667161 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:01:50.667161 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:01:50.667161 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:01:50.667161 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:01:50.667161 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:01:50.667161 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:01:50.667161 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 12:01:50.667161 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 12:01:50.667161 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 12:01:50.668183 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 12:01:50.668183 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:01:50.668183 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:01:50.668183 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:01:50.668183 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:01:50.668183 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:01:50.668183 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:01:50.668183 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:01:50.668183 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jul 25 12:01:50.668183 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:01:50.668183 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:01:50.668183 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:01:50.668183 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:01:50.668183 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:01:50.668183 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:01:50.668183 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:01:50.668183 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:01:50.669044 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 12:01:50.669044 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 12:01:50.669044 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:01:50.669044 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jul 25 12:01:50.671005 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 232/462] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jul 25 12:01:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:01:50.677626 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcfcc35-ed91-49d6-a24f-15d401daa0ca None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:50.678188 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcfcc35-ed91-49d6-a24f-15d401daa0ca None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:50.678662 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcfcc35-ed91-49d6-a24f-15d401daa0ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:50.697904 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:50.699928 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-abcfcc35-ed91-49d6-a24f-15d401daa0ca None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:01:50.701336 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-abcfcc35-ed91-49d6-a24f-15d401daa0ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:01:50.757261 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533a9489-c91c-4d20-9a3d-dc90313f1ce8 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:50.780617 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 233/463] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 25 12:01:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 12:01:50.781338 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcfcc35-ed91-49d6-a24f-15d401daa0ca None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:50.802545 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03715011-2ce5-4fe6-ab3f-455f54666a14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:50.803409 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03715011-2ce5-4fe6-ab3f-455f54666a14 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:50.803766 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03715011-2ce5-4fe6-ab3f-455f54666a14 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:50.804052 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03715011-2ce5-4fe6-ab3f-455f54666a14 None admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:50.804882 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03715011-2ce5-4fe6-ab3f-455f54666a14 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:01:50.805125 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03715011-2ce5-4fe6-ab3f-455f54666a14 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:50.806511 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03715011-2ce5-4fe6-ab3f-455f54666a14 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:50.818696 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 231/464] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jul 25 12:01:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:01:50.820211 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c4472f-12a1-4886-8ea2-d475bb9a2d33 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:50.820514 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c4472f-12a1-4886-8ea2-d475bb9a2d33 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:50.820800 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c4472f-12a1-4886-8ea2-d475bb9a2d33 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:50.822206 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 232/465] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 25 12:01:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:01:50.827003 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c4472f-12a1-4886-8ea2-d475bb9a2d33 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:50.857940 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:50.906939 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcfcc35-ed91-49d6-a24f-15d401daa0ca None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:50.938739 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c4472f-12a1-4886-8ea2-d475bb9a2d33 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:50.943398 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15eca20e-f723-4609-91fa-1196c658423e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:50.944128 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15eca20e-f723-4609-91fa-1196c658423e None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:50.944418 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15eca20e-f723-4609-91fa-1196c658423e None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:50.944705 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15eca20e-f723-4609-91fa-1196c658423e None admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:50.945431 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15eca20e-f723-4609-91fa-1196c658423e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:01:50.945749 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15eca20e-f723-4609-91fa-1196c658423e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:50.947114 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15eca20e-f723-4609-91fa-1196c658423e None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:50.948350 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:50.969380 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 234/466] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jul 25 12:01:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:01:50.969380 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15eca20e-f723-4609-91fa-1196c658423e None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:50.979801 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af3ee17-b57e-4578-b8a0-c1aadcc3c164 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:50.980601 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af3ee17-b57e-4578-b8a0-c1aadcc3c164 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:50.980893 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af3ee17-b57e-4578-b8a0-c1aadcc3c164 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:50.981175 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af3ee17-b57e-4578-b8a0-c1aadcc3c164 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:50.992843 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af3ee17-b57e-4578-b8a0-c1aadcc3c164 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:50.994290 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af3ee17-b57e-4578-b8a0-c1aadcc3c164 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:51.011366 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-8af3ee17-b57e-4578-b8a0-c1aadcc3c164 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:01:51.011366 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:01:51.011366 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:01:51.011366 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:01:51.011366 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:01:51.011366 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:01:51.011366 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:01:51.011366 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:01:51.011366 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:01:51.011366 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:01:51.011366 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:01:51.011366 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:01:51.011366 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 12:01:51.011366 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 12:01:51.011366 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 12:01:51.012341 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 12:01:51.012341 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:01:51.012341 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:01:51.012341 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:01:51.012341 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:01:51.012341 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:01:51.012341 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:01:51.012341 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:01:51.012341 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 25 12:01:51.012341 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:01:51.012341 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:01:51.012341 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:01:51.012341 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:01:51.012341 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:01:51.012341 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:01:51.012341 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:01:51.012341 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:01:51.018833 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 12:01:51.018833 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 12:01:51.018833 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:01:51.018833 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 25 12:01:51.018833 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 233/467] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jul 25 12:01:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:01:51.018833 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0339a236-fc1b-4c54-b259-076409558799 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:51.018833 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0339a236-fc1b-4c54-b259-076409558799 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:51.018833 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0339a236-fc1b-4c54-b259-076409558799 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:51.034890 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-0339a236-fc1b-4c54-b259-076409558799 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:01:51.036116 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-0339a236-fc1b-4c54-b259-076409558799 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:01:51.086051 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15eca20e-f723-4609-91fa-1196c658423e None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:51.104355 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:51.104585 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 234/468] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 25 12:01:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 12:01:51.105407 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0339a236-fc1b-4c54-b259-076409558799 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:51.147257 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f616594d-61ec-4604-b3ad-3a53061bf4ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:51.148015 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f616594d-61ec-4604-b3ad-3a53061bf4ec None admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:51.148679 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f616594d-61ec-4604-b3ad-3a53061bf4ec None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:51.148973 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f616594d-61ec-4604-b3ad-3a53061bf4ec None admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:51.150007 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f616594d-61ec-4604-b3ad-3a53061bf4ec None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:51.151470 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f616594d-61ec-4604-b3ad-3a53061bf4ec None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:51.187585 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 235/469] 10.222.0.42 () {60 vars in 1132 bytes} [Mon Jul 25 12:01:50 2022] POST /identity/v3/users => generated 248 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:01:51.187585 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f616594d-61ec-4604-b3ad-3a53061bf4ec None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:51.229769 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0339a236-fc1b-4c54-b259-076409558799 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:51.241469 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:51.278690 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f0e5c5-bf99-4dc6-8a65-457c32a21808 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:51.279306 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f0e5c5-bf99-4dc6-8a65-457c32a21808 None admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:51.279622 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f0e5c5-bf99-4dc6-8a65-457c32a21808 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:51.279908 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f0e5c5-bf99-4dc6-8a65-457c32a21808 None admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:51.280779 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f0e5c5-bf99-4dc6-8a65-457c32a21808 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:51.282418 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f0e5c5-bf99-4dc6-8a65-457c32a21808 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:51.316084 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 235/470] 10.222.0.42 () {60 vars in 1132 bytes} [Mon Jul 25 12:01:51 2022] POST /identity/v3/users => generated 246 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:01:51.317237 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f0e5c5-bf99-4dc6-8a65-457c32a21808 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:51.330057 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f616594d-61ec-4604-b3ad-3a53061bf4ec None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:51.374599 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da8dcb9-7dad-4654-ac03-918231563bd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:51.375563 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da8dcb9-7dad-4654-ac03-918231563bd3 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:51.376023 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da8dcb9-7dad-4654-ac03-918231563bd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:51.376487 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da8dcb9-7dad-4654-ac03-918231563bd3 None admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:51.378695 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da8dcb9-7dad-4654-ac03-918231563bd3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:01:51.379282 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da8dcb9-7dad-4654-ac03-918231563bd3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:51.380928 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da8dcb9-7dad-4654-ac03-918231563bd3 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:51.395856 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 236/471] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jul 25 12:01:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:01:51.402417 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da8dcb9-7dad-4654-ac03-918231563bd3 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:51.457208 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:51.484573 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f0e5c5-bf99-4dc6-8a65-457c32a21808 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:51.537892 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aba1732-1d99-44ac-bf4a-659d82cf7535 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:51.538989 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aba1732-1d99-44ac-bf4a-659d82cf7535 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:51.539279 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aba1732-1d99-44ac-bf4a-659d82cf7535 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:51.539659 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aba1732-1d99-44ac-bf4a-659d82cf7535 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:51.548175 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da8dcb9-7dad-4654-ac03-918231563bd3 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:51.552998 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aba1732-1d99-44ac-bf4a-659d82cf7535 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:51.554473 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aba1732-1d99-44ac-bf4a-659d82cf7535 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:51.561381 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:51.576476 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-8aba1732-1d99-44ac-bf4a-659d82cf7535 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:01:51.576476 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:01:51.576476 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:01:51.576476 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:01:51.576476 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:01:51.576476 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:01:51.576476 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:01:51.576476 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:01:51.576476 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:01:51.576476 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:01:51.576476 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:01:51.576476 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:01:51.576476 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 12:01:51.576476 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 12:01:51.576476 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 12:01:51.577378 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 12:01:51.577378 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:01:51.577378 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:01:51.577378 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:01:51.577378 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:01:51.577378 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:01:51.577378 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:01:51.577378 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:01:51.577378 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 25 12:01:51.577378 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:01:51.577378 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:01:51.577378 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:01:51.577378 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:01:51.577378 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:01:51.577378 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:01:51.577378 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:01:51.577378 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:01:51.578212 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 12:01:51.578212 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 12:01:51.578212 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:01:51.578212 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 25 12:01:51.578212 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 236/472] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jul 25 12:01:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:01:51.584315 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aba1732-1d99-44ac-bf4a-659d82cf7535 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:51.592860 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cd99ba-0a04-49a4-877c-40a0ad46021e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:51.593766 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cd99ba-0a04-49a4-877c-40a0ad46021e None admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:51.594190 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cd99ba-0a04-49a4-877c-40a0ad46021e None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:51.595074 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cd99ba-0a04-49a4-877c-40a0ad46021e None admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:51.595974 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69cd99ba-0a04-49a4-877c-40a0ad46021e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:51.598218 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69cd99ba-0a04-49a4-877c-40a0ad46021e None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:51.623669 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 237/473] 10.222.0.42 () {60 vars in 1132 bytes} [Mon Jul 25 12:01:51 2022] POST /identity/v3/users => generated 248 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:01:51.658508 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adfaced-0b90-4ddb-b44f-fff6fe00ca55 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:51.658867 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adfaced-0b90-4ddb-b44f-fff6fe00ca55 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:51.659126 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adfaced-0b90-4ddb-b44f-fff6fe00ca55 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:51.660651 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 238/474] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 25 12:01:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 12:01:51.666880 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1db02a-e6ed-4fea-80cf-1cbf7e108a7a None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:51.667490 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1db02a-e6ed-4fea-80cf-1cbf7e108a7a None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:51.668004 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1db02a-e6ed-4fea-80cf-1cbf7e108a7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:51.691926 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-6a1db02a-e6ed-4fea-80cf-1cbf7e108a7a None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:01:51.694192 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-6a1db02a-e6ed-4fea-80cf-1cbf7e108a7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:01:51.723382 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aba1732-1d99-44ac-bf4a-659d82cf7535 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:51.777915 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7ee1f2-3a9a-4d0c-9b0f-584bf2dde246 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:51.778994 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7ee1f2-3a9a-4d0c-9b0f-584bf2dde246 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:51.779637 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7ee1f2-3a9a-4d0c-9b0f-584bf2dde246 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:51.779927 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7ee1f2-3a9a-4d0c-9b0f-584bf2dde246 None admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:51.780813 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7ee1f2-3a9a-4d0c-9b0f-584bf2dde246 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:01:51.781246 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7ee1f2-3a9a-4d0c-9b0f-584bf2dde246 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:51.782869 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7ee1f2-3a9a-4d0c-9b0f-584bf2dde246 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:51.797842 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 237/475] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jul 25 12:01:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:01:51.800711 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 239/476] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 25 12:01:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 25 12:01:51.803462 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7ee1f2-3a9a-4d0c-9b0f-584bf2dde246 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:51.826944 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1db02a-e6ed-4fea-80cf-1cbf7e108a7a None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:51.850695 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:51.874347 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:51.930754 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7ee1f2-3a9a-4d0c-9b0f-584bf2dde246 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:51.967355 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1db02a-e6ed-4fea-80cf-1cbf7e108a7a None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:51.969323 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072c7273-82d2-4ced-9a5c-ec031b1a61e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:51.970006 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072c7273-82d2-4ced-9a5c-ec031b1a61e3 None admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:51.970297 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072c7273-82d2-4ced-9a5c-ec031b1a61e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:51.970575 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072c7273-82d2-4ced-9a5c-ec031b1a61e3 None admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:51.971337 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072c7273-82d2-4ced-9a5c-ec031b1a61e3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:51.972837 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072c7273-82d2-4ced-9a5c-ec031b1a61e3 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:51.999584 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 238/477] 10.222.0.42 () {60 vars in 1132 bytes} [Mon Jul 25 12:01:51 2022] POST /identity/v3/users => generated 249 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 12:01:52.033086 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acaaa948-cdfe-417b-b2d5-76d2bb64a62f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:52.034732 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acaaa948-cdfe-417b-b2d5-76d2bb64a62f None admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:52.035167 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acaaa948-cdfe-417b-b2d5-76d2bb64a62f None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:52.035711 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acaaa948-cdfe-417b-b2d5-76d2bb64a62f None admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:52.036680 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acaaa948-cdfe-417b-b2d5-76d2bb64a62f None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:52.038499 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acaaa948-cdfe-417b-b2d5-76d2bb64a62f None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:52.050929 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.sql.core [None req-acaaa948-cdfe-417b-b2d5-76d2bb64a62f None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 25 12:01:52.050929 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 25 12:01:52.050929 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [parameters: {'id': '69e122a672904c68a7c6dd29ff777ca6', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 25 12:01:52.050929 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80298) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 25 12:01:52.069863 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-acaaa948-cdfe-417b-b2d5-76d2bb64a62f None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 25 12:01:52.069863 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:01:52.069863 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 25 12:01:52.069863 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 25 12:01:52.069863 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 25 12:01:52.069863 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 25 12:01:52.069863 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 25 12:01:52.069863 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application result = self._query(query) Jul 25 12:01:52.069863 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 25 12:01:52.069863 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application conn.query(q) Jul 25 12:01:52.069863 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 25 12:01:52.069863 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 25 12:01:52.069863 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 25 12:01:52.069863 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application result.read() Jul 25 12:01:52.069863 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 25 12:01:52.071004 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 25 12:01:52.071004 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 25 12:01:52.071004 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 25 12:01:52.071004 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 25 12:01:52.071004 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 25 12:01:52.071004 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 25 12:01:52.071004 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 25 12:01:52.071004 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 25 12:01:52.071004 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jul 25 12:01:52.071004 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 25 12:01:52.071004 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jul 25 12:01:52.071004 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:01:52.071004 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 25 12:01:52.071004 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 25 12:01:52.071004 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jul 25 12:01:52.071004 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return ref.to_dict() Jul 25 12:01:52.071004 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 25 12:01:52.071004 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application next(self.gen) Jul 25 12:01:52.072175 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 25 12:01:52.072175 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application yield resource Jul 25 12:01:52.072175 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 25 12:01:52.072175 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application next(self.gen) Jul 25 12:01:52.072175 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 25 12:01:52.072175 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application self.session.rollback() Jul 25 12:01:52.072175 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 25 12:01:52.072175 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application self.force_reraise() Jul 25 12:01:52.072175 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 25 12:01:52.072175 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise self.value Jul 25 12:01:52.072175 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 25 12:01:52.072175 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 25 12:01:52.072175 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 25 12:01:52.072175 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application session.commit() Jul 25 12:01:52.072175 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 25 12:01:52.072175 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 25 12:01:52.072175 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 25 12:01:52.073052 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application self._prepare_impl() Jul 25 12:01:52.073052 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 25 12:01:52.073052 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application self.session.flush() Jul 25 12:01:52.073052 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 25 12:01:52.073052 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application self._flush(objects) Jul 25 12:01:52.073052 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 25 12:01:52.073052 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 25 12:01:52.073052 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 25 12:01:52.073052 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application compat.raise_( Jul 25 12:01:52.073052 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 25 12:01:52.073052 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception Jul 25 12:01:52.073052 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 25 12:01:52.073052 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application flush_context.execute() Jul 25 12:01:52.073052 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 25 12:01:52.073052 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rec.execute(self) Jul 25 12:01:52.073052 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 25 12:01:52.073052 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 25 12:01:52.074220 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 25 12:01:52.074220 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 25 12:01:52.074220 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 25 12:01:52.074220 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 25 12:01:52.074220 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 25 12:01:52.074220 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 25 12:01:52.074220 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 25 12:01:52.074220 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 25 12:01:52.074220 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 25 12:01:52.074220 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 25 12:01:52.074220 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 25 12:01:52.074220 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 25 12:01:52.074220 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 25 12:01:52.074220 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 25 12:01:52.074220 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 25 12:01:52.074220 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception Jul 25 12:01:52.075389 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 25 12:01:52.075389 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 25 12:01:52.075389 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 25 12:01:52.075389 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 25 12:01:52.075389 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 25 12:01:52.075389 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application result = self._query(query) Jul 25 12:01:52.075389 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 25 12:01:52.075389 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application conn.query(q) Jul 25 12:01:52.075389 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 25 12:01:52.075389 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 25 12:01:52.075389 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 25 12:01:52.075389 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application result.read() Jul 25 12:01:52.075389 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 25 12:01:52.075389 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 25 12:01:52.075389 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 25 12:01:52.075389 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 25 12:01:52.076522 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 25 12:01:52.076522 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 25 12:01:52.076522 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 25 12:01:52.076522 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 25 12:01:52.076522 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 25 12:01:52.076522 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 25 12:01:52.076522 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [parameters: {'id': '69e122a672904c68a7c6dd29ff777ca6', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 25 12:01:52.076522 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 25 12:01:52.076522 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jul 25 12:01:52.076522 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 12:01:52.076522 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jul 25 12:01:52.076522 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:01:52.076522 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:01:52.076522 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:01:52.076522 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:01:52.076522 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:01:52.077312 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:01:52.077312 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:01:52.077312 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:01:52.077312 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:01:52.077312 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:01:52.077312 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:01:52.077312 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jul 25 12:01:52.077312 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jul 25 12:01:52.077312 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:01:52.077312 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:01:52.077312 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jul 25 12:01:52.077312 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jul 25 12:01:52.077312 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 25 12:01:52.077312 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 25 12:01:52.077312 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 25 12:01:52.077312 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jul 25 12:01:52.080394 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 240/478] 10.222.0.42 () {60 vars in 1131 bytes} [Mon Jul 25 12:01:51 2022] POST /identity/v3/roles => generated 144 bytes in 254 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jul 25 12:01:52.086717 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072c7273-82d2-4ced-9a5c-ec031b1a61e3 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:52.214978 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072c7273-82d2-4ced-9a5c-ec031b1a61e3 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:52.231116 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:52.257281 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5e4465-89e6-44a7-aafa-00eb21e12c80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:52.258427 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5e4465-89e6-44a7-aafa-00eb21e12c80 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:52.258907 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5e4465-89e6-44a7-aafa-00eb21e12c80 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:52.259396 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5e4465-89e6-44a7-aafa-00eb21e12c80 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:52.271832 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5e4465-89e6-44a7-aafa-00eb21e12c80 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 12:01:52.279975 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-af5e4465-89e6-44a7-aafa-00eb21e12c80 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 12:01:52.279975 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:01:52.279975 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:01:52.279975 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:01:52.279975 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:01:52.279975 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:01:52.279975 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:01:52.279975 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:01:52.279975 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:01:52.279975 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:01:52.279975 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:01:52.279975 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:01:52.279975 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 12:01:52.279975 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 12:01:52.279975 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 12:01:52.279975 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 12:01:52.280861 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 12:01:52.280861 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 12:01:52.280861 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 12:01:52.280861 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 12:01:52.280861 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:01:52.280861 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:01:52.280861 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:01:52.280861 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:01:52.280861 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:01:52.280861 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:01:52.280861 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:01:52.280861 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 25 12:01:52.280861 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:01:52.280861 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:01:52.280861 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:01:52.280861 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:01:52.281649 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:01:52.281649 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:01:52.281649 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:01:52.281649 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:01:52.281649 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 12:01:52.281649 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 12:01:52.281649 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 12:01:52.281649 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 12:01:52.281649 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 12:01:52.281649 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 12:01:52.281649 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 12:01:52.281649 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 25 12:01:52.284346 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 239/479] 10.222.0.42 () {56 vars in 1114 bytes} [Mon Jul 25 12:01:52 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:01:52.290311 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acaaa948-cdfe-417b-b2d5-76d2bb64a62f None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:52.386346 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:52.407958 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acaaa948-cdfe-417b-b2d5-76d2bb64a62f None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:52.452585 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bcfabe-5d2d-434a-b4ef-08281d29739d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:52.453270 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bcfabe-5d2d-434a-b4ef-08281d29739d None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:52.453562 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bcfabe-5d2d-434a-b4ef-08281d29739d None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:52.453850 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bcfabe-5d2d-434a-b4ef-08281d29739d None admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:52.454615 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47bcfabe-5d2d-434a-b4ef-08281d29739d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:01:52.454934 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47bcfabe-5d2d-434a-b4ef-08281d29739d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:52.456338 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47bcfabe-5d2d-434a-b4ef-08281d29739d None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:52.465174 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 241/480] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Jul 25 12:01:52 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:01:53.719417 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dc59c3-9ba2-42f5-8aaf-e43d1b64869e None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:53.719962 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dc59c3-9ba2-42f5-8aaf-e43d1b64869e None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:53.720039 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dc59c3-9ba2-42f5-8aaf-e43d1b64869e None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:53.721702 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 240/481] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 25 12:01:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:01:53.727278 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2298f12-5c18-47fa-add6-21fccf1a0899 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:53.727815 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2298f12-5c18-47fa-add6-21fccf1a0899 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:53.728058 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2298f12-5c18-47fa-add6-21fccf1a0899 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:53.763676 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-e2298f12-5c18-47fa-add6-21fccf1a0899 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:01:53.764897 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-e2298f12-5c18-47fa-add6-21fccf1a0899 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:01:53.867205 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 242/482] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 25 12:01:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 25 12:01:53.913107 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71dc59c3-9ba2-42f5-8aaf-e43d1b64869e None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:53.989720 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2abb921-3a27-4a34-acee-3e8000c184fe None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:53.990012 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2abb921-3a27-4a34-acee-3e8000c184fe None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:53.990207 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2abb921-3a27-4a34-acee-3e8000c184fe None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:53.991738 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 243/483] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 25 12:01:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 12:01:53.996393 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5579b991-94b1-4c4f-bff9-4375ef765360 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:53.996682 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5579b991-94b1-4c4f-bff9-4375ef765360 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:53.996964 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5579b991-94b1-4c4f-bff9-4375ef765360 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:54.016201 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-5579b991-94b1-4c4f-bff9-4375ef765360 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:01:54.017397 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-5579b991-94b1-4c4f-bff9-4375ef765360 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:01:54.062638 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71dc59c3-9ba2-42f5-8aaf-e43d1b64869e None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:54.140185 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efb212b-e582-48bf-9600-001ccc974a0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:54.141385 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efb212b-e582-48bf-9600-001ccc974a0b None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:54.141876 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efb212b-e582-48bf-9600-001ccc974a0b None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:54.142354 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efb212b-e582-48bf-9600-001ccc974a0b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:54.162193 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 244/484] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 25 12:01:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 12:01:54.162944 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6efb212b-e582-48bf-9600-001ccc974a0b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 12:01:54.171448 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-6efb212b-e582-48bf-9600-001ccc974a0b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 25 12:01:54.171448 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:01:54.171448 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:01:54.171448 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:01:54.171448 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:01:54.171448 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:01:54.171448 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:01:54.171448 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:01:54.171448 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:01:54.171448 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:01:54.171448 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:01:54.171448 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:01:54.171448 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 12:01:54.171448 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 12:01:54.171448 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 12:01:54.171448 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 12:01:54.172371 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 12:01:54.172371 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 12:01:54.172371 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 12:01:54.172371 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 12:01:54.172371 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:01:54.172371 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:01:54.172371 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:01:54.172371 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:01:54.172371 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:01:54.172371 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:01:54.172371 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:01:54.172371 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 25 12:01:54.172371 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:01:54.172371 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:01:54.172371 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:01:54.172371 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:01:54.173166 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:01:54.173166 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:01:54.173166 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:01:54.173166 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:01:54.173166 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 12:01:54.173166 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 12:01:54.173166 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 12:01:54.173166 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 12:01:54.173166 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 12:01:54.173166 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 12:01:54.173166 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 25 12:01:54.173166 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 25 12:01:54.176341 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 241/485] 10.222.0.42 () {56 vars in 1102 bytes} [Mon Jul 25 12:01:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:01:54.181974 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5579b991-94b1-4c4f-bff9-4375ef765360 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:54.188520 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efb212b-e582-48bf-9600-001ccc974a0b None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:54.196678 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:54.235953 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:54.298119 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efb212b-e582-48bf-9600-001ccc974a0b None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:54.337015 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fa070d-cc61-4a36-8430-a7cadab1713a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:54.337826 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fa070d-cc61-4a36-8430-a7cadab1713a None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:54.338115 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fa070d-cc61-4a36-8430-a7cadab1713a None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:54.338400 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fa070d-cc61-4a36-8430-a7cadab1713a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:54.349676 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95fa070d-cc61-4a36-8430-a7cadab1713a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 12:01:54.357840 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-95fa070d-cc61-4a36-8430-a7cadab1713a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 25 12:01:54.357840 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:01:54.357840 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:01:54.357840 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:01:54.357840 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:01:54.357840 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:01:54.357840 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:01:54.357840 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:01:54.357840 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:01:54.357840 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:01:54.357840 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:01:54.357840 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:01:54.357840 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 12:01:54.357840 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 12:01:54.357840 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 12:01:54.357840 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 12:01:54.358748 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 12:01:54.358748 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 12:01:54.358748 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 12:01:54.358748 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 12:01:54.358748 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:01:54.358748 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:01:54.358748 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:01:54.358748 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:01:54.358748 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:01:54.358748 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:01:54.358748 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:01:54.358748 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 25 12:01:54.358748 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:01:54.358748 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:01:54.358748 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:01:54.358748 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:01:54.359567 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:01:54.359567 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:01:54.359567 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:01:54.359567 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:01:54.359567 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 12:01:54.359567 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 12:01:54.359567 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 12:01:54.359567 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 12:01:54.359567 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 12:01:54.359567 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 12:01:54.359567 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 25 12:01:54.359567 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 25 12:01:54.359567 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 242/486] 10.222.0.42 () {56 vars in 1102 bytes} [Mon Jul 25 12:01:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:01:54.366535 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fa070d-cc61-4a36-8430-a7cadab1713a None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:54.463140 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5579b991-94b1-4c4f-bff9-4375ef765360 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:54.479353 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fa070d-cc61-4a36-8430-a7cadab1713a None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:54.481457 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:54.514022 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59d8d55-4027-4458-b0bf-1d05328792ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:54.514705 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59d8d55-4027-4458-b0bf-1d05328792ff None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:54.514994 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59d8d55-4027-4458-b0bf-1d05328792ff None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:54.515470 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59d8d55-4027-4458-b0bf-1d05328792ff None admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:54.516570 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59d8d55-4027-4458-b0bf-1d05328792ff None admin] RBAC: Adding query filter params (name=service) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:01:54.516886 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59d8d55-4027-4458-b0bf-1d05328792ff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:54.518501 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59d8d55-4027-4458-b0bf-1d05328792ff None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:54.522179 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a3da37-68b8-4bbf-8879-f42c2000dfcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:54.522948 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a3da37-68b8-4bbf-8879-f42c2000dfcc None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:54.523296 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a3da37-68b8-4bbf-8879-f42c2000dfcc None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:54.523618 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a3da37-68b8-4bbf-8879-f42c2000dfcc None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:54.524492 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a3da37-68b8-4bbf-8879-f42c2000dfcc None admin] RBAC: Adding query filter params (name=service) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:01:54.524870 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a3da37-68b8-4bbf-8879-f42c2000dfcc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:54.526925 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a3da37-68b8-4bbf-8879-f42c2000dfcc None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:54.529612 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 245/487] 10.222.0.42 () {56 vars in 1103 bytes} [Mon Jul 25 12:01:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 348 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:01:54.530922 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba4da1c-8587-4955-94d7-ded4b3037180 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:54.531215 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba4da1c-8587-4955-94d7-ded4b3037180 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:54.531596 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba4da1c-8587-4955-94d7-ded4b3037180 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:54.533449 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 246/488] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 25 12:01:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:01:54.536798 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 243/489] 10.222.0.42 () {56 vars in 1103 bytes} [Mon Jul 25 12:01:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 12:01:54.540124 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba4da1c-8587-4955-94d7-ded4b3037180 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:54.543013 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8deb58-b6e8-466f-9acb-5d2d93a8d5ee None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:54.543418 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8deb58-b6e8-466f-9acb-5d2d93a8d5ee None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:54.543990 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8deb58-b6e8-466f-9acb-5d2d93a8d5ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:54.567086 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-dd8deb58-b6e8-466f-9acb-5d2d93a8d5ee None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:01:54.568631 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-dd8deb58-b6e8-466f-9acb-5d2d93a8d5ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:01:54.629922 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:54.653268 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 244/490] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 25 12:01:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 12:01:54.654318 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8deb58-b6e8-466f-9acb-5d2d93a8d5ee None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:54.671978 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba4da1c-8587-4955-94d7-ded4b3037180 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:54.707158 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b04ec60-102b-4170-9b8f-e3dc4be4f733 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:54.707985 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b04ec60-102b-4170-9b8f-e3dc4be4f733 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:54.708272 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b04ec60-102b-4170-9b8f-e3dc4be4f733 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:54.708551 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b04ec60-102b-4170-9b8f-e3dc4be4f733 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:54.719418 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b04ec60-102b-4170-9b8f-e3dc4be4f733 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:54.721531 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b04ec60-102b-4170-9b8f-e3dc4be4f733 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:54.739391 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-0b04ec60-102b-4170-9b8f-e3dc4be4f733 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:01:54.739391 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:01:54.739391 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:01:54.739391 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:01:54.739391 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:01:54.739391 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:01:54.739391 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:01:54.739391 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:01:54.739391 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:01:54.739391 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:01:54.739391 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:01:54.739391 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:01:54.739391 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 12:01:54.739391 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 12:01:54.739391 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 12:01:54.741096 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 12:01:54.741096 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:01:54.741096 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:01:54.741096 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:01:54.741096 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:01:54.741096 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:01:54.741096 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:01:54.741096 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:01:54.741096 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jul 25 12:01:54.741096 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:01:54.741096 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:01:54.741096 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:01:54.741096 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:01:54.741096 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:01:54.741096 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:01:54.741096 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:01:54.741096 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:01:54.742327 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 12:01:54.742327 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 12:01:54.742327 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:01:54.742327 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jul 25 12:01:54.742327 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 247/491] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jul 25 12:01:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:01:54.742327 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b04ec60-102b-4170-9b8f-e3dc4be4f733 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:54.751669 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:54.769395 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8deb58-b6e8-466f-9acb-5d2d93a8d5ee None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:54.809552 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63781e0-a261-40a3-91b8-2d2084812557 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:54.810385 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63781e0-a261-40a3-91b8-2d2084812557 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:54.810673 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63781e0-a261-40a3-91b8-2d2084812557 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:54.810955 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63781e0-a261-40a3-91b8-2d2084812557 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:54.822621 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63781e0-a261-40a3-91b8-2d2084812557 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:54.823975 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63781e0-a261-40a3-91b8-2d2084812557 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:54.845907 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-a63781e0-a261-40a3-91b8-2d2084812557 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:01:54.845907 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:01:54.845907 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:01:54.845907 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:01:54.845907 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:01:54.845907 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:01:54.845907 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:01:54.845907 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:01:54.845907 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:01:54.845907 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:01:54.845907 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:01:54.845907 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:01:54.845907 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 12:01:54.845907 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 12:01:54.845907 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 12:01:54.847424 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 12:01:54.847424 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:01:54.847424 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:01:54.847424 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:01:54.847424 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:01:54.847424 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:01:54.847424 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:01:54.847424 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:01:54.847424 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 25 12:01:54.847424 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:01:54.847424 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:01:54.847424 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:01:54.847424 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:01:54.847424 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:01:54.847424 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:01:54.847424 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:01:54.847424 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:01:54.848319 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 12:01:54.848319 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 12:01:54.848319 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:01:54.848319 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 25 12:01:54.852163 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 245/492] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jul 25 12:01:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:01:54.852962 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ac9459-63b7-4e36-a877-9d48a688c340 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:54.853425 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ac9459-63b7-4e36-a877-9d48a688c340 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:54.853553 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ac9459-63b7-4e36-a877-9d48a688c340 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:54.854789 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 246/493] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 25 12:01:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:01:54.855474 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ac9459-63b7-4e36-a877-9d48a688c340 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:54.868327 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b04ec60-102b-4170-9b8f-e3dc4be4f733 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:54.887130 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:54.906805 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488c8f23-6256-403f-ab88-dcc5b5f82407 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:54.907650 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488c8f23-6256-403f-ab88-dcc5b5f82407 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:54.907950 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488c8f23-6256-403f-ab88-dcc5b5f82407 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:54.908238 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488c8f23-6256-403f-ab88-dcc5b5f82407 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:54.920098 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488c8f23-6256-403f-ab88-dcc5b5f82407 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 12:01:54.927994 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-488c8f23-6256-403f-ab88-dcc5b5f82407 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 25 12:01:54.927994 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:01:54.927994 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:01:54.927994 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:01:54.927994 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:01:54.927994 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:01:54.927994 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:01:54.927994 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:01:54.927994 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:01:54.927994 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:01:54.927994 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:01:54.927994 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:01:54.927994 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 12:01:54.927994 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 12:01:54.927994 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 12:01:54.927994 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 12:01:54.928980 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 12:01:54.928980 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 12:01:54.928980 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 12:01:54.928980 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 12:01:54.928980 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:01:54.928980 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:01:54.928980 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:01:54.928980 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:01:54.928980 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:01:54.928980 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:01:54.928980 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:01:54.928980 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jul 25 12:01:54.928980 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:01:54.928980 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:01:54.928980 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:01:54.928980 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:01:54.929755 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:01:54.929755 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:01:54.929755 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:01:54.929755 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:01:54.929755 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 12:01:54.929755 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 12:01:54.929755 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 12:01:54.929755 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 12:01:54.929755 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 12:01:54.929755 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 12:01:54.929755 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 25 12:01:54.929755 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jul 25 12:01:54.929755 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 248/494] 10.222.0.42 () {56 vars in 1102 bytes} [Mon Jul 25 12:01:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:01:54.930418 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488c8f23-6256-403f-ab88-dcc5b5f82407 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:54.970870 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ac9459-63b7-4e36-a877-9d48a688c340 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:55.007004 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223ae0c9-d105-4bbb-a82c-a1f1be216a99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:55.007724 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223ae0c9-d105-4bbb-a82c-a1f1be216a99 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:55.008017 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223ae0c9-d105-4bbb-a82c-a1f1be216a99 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:55.008304 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223ae0c9-d105-4bbb-a82c-a1f1be216a99 None admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:55.009035 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223ae0c9-d105-4bbb-a82c-a1f1be216a99 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:01:55.009353 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223ae0c9-d105-4bbb-a82c-a1f1be216a99 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:55.010741 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223ae0c9-d105-4bbb-a82c-a1f1be216a99 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:55.018424 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:55.024546 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 247/495] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jul 25 12:01:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:01:55.025778 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac52dd3c-d4cc-45ea-873b-7d6d11053f6b None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:55.026076 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac52dd3c-d4cc-45ea-873b-7d6d11053f6b None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:55.026363 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac52dd3c-d4cc-45ea-873b-7d6d11053f6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:55.045548 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-ac52dd3c-d4cc-45ea-873b-7d6d11053f6b None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:01:55.046742 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-ac52dd3c-d4cc-45ea-873b-7d6d11053f6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:01:55.051751 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488c8f23-6256-403f-ab88-dcc5b5f82407 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:55.087879 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ad6ff3-a774-4562-91fb-e0e02a4d078f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:55.088461 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ad6ff3-a774-4562-91fb-e0e02a4d078f None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:55.088796 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ad6ff3-a774-4562-91fb-e0e02a4d078f None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:55.089082 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ad6ff3-a774-4562-91fb-e0e02a4d078f None admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:55.089830 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ad6ff3-a774-4562-91fb-e0e02a4d078f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:01:55.090142 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ad6ff3-a774-4562-91fb-e0e02a4d078f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:55.091517 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ad6ff3-a774-4562-91fb-e0e02a4d078f None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:55.114802 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 249/496] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jul 25 12:01:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:01:55.115684 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ad6ff3-a774-4562-91fb-e0e02a4d078f None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:55.131641 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 248/497] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 25 12:01:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 12:01:55.131641 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac52dd3c-d4cc-45ea-873b-7d6d11053f6b None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:55.171468 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:55.235343 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ad6ff3-a774-4562-91fb-e0e02a4d078f None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:55.241058 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac52dd3c-d4cc-45ea-873b-7d6d11053f6b None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:55.266382 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:55.269344 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afab9a4-76d0-4079-a592-7e1f65ebb618 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:55.270020 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afab9a4-76d0-4079-a592-7e1f65ebb618 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:55.270308 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afab9a4-76d0-4079-a592-7e1f65ebb618 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:55.270587 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afab9a4-76d0-4079-a592-7e1f65ebb618 None admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:55.271365 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3afab9a4-76d0-4079-a592-7e1f65ebb618 None admin] RBAC: Adding query filter params (name=service) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:01:55.271702 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3afab9a4-76d0-4079-a592-7e1f65ebb618 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:55.273071 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3afab9a4-76d0-4079-a592-7e1f65ebb618 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:55.274703 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686dd372-5055-453e-93c0-b7aa3845b761 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:55.275525 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686dd372-5055-453e-93c0-b7aa3845b761 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:55.275851 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686dd372-5055-453e-93c0-b7aa3845b761 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:55.276133 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686dd372-5055-453e-93c0-b7aa3845b761 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:55.280932 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 250/498] 10.222.0.42 () {56 vars in 1103 bytes} [Mon Jul 25 12:01:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:01:55.281707 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afab9a4-76d0-4079-a592-7e1f65ebb618 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:55.284172 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686dd372-5055-453e-93c0-b7aa3845b761 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:55.285613 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686dd372-5055-453e-93c0-b7aa3845b761 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:55.302228 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-686dd372-5055-453e-93c0-b7aa3845b761 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 25 12:01:55.302228 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:01:55.302228 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:01:55.302228 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:01:55.302228 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:01:55.302228 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:01:55.302228 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:01:55.302228 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:01:55.302228 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:01:55.302228 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:01:55.302228 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:01:55.302228 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:01:55.302228 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 12:01:55.302228 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 12:01:55.302228 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 12:01:55.302228 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 12:01:55.303131 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:01:55.303131 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:01:55.303131 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 12:01:55.303131 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 12:01:55.303131 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 12:01:55.303131 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 12:01:55.303131 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:01:55.303131 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:01:55.303131 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:01:55.303131 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:01:55.303131 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:01:55.303131 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 25 12:01:55.303131 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:01:55.303131 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:01:55.303131 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:01:55.303131 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:01:55.303131 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:01:55.303131 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:01:55.308870 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:01:55.308870 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:01:55.308870 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 12:01:55.308870 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 12:01:55.308870 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 12:01:55.308870 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 12:01:55.308870 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 12:01:55.308870 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 12:01:55.308870 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 25 12:01:55.308870 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 25 12:01:55.308870 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 249/499] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Jul 25 12:01:55 2022] GET /identity/v3/users/glance => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:01:55.308870 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686dd372-5055-453e-93c0-b7aa3845b761 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:55.381178 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:55.384111 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afab9a4-76d0-4079-a592-7e1f65ebb618 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:55.401571 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686dd372-5055-453e-93c0-b7aa3845b761 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:55.418062 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85702336-e7e9-4852-84ba-fc3aeeb0a7b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:55.418849 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85702336-e7e9-4852-84ba-fc3aeeb0a7b3 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:55.419144 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85702336-e7e9-4852-84ba-fc3aeeb0a7b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:55.419424 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85702336-e7e9-4852-84ba-fc3aeeb0a7b3 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:55.427436 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85702336-e7e9-4852-84ba-fc3aeeb0a7b3 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:55.428908 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85702336-e7e9-4852-84ba-fc3aeeb0a7b3 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:55.436147 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44bbf7f-cca7-43b0-9954-0a382ef69f05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:55.436933 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44bbf7f-cca7-43b0-9954-0a382ef69f05 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:55.437219 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44bbf7f-cca7-43b0-9954-0a382ef69f05 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:55.437498 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44bbf7f-cca7-43b0-9954-0a382ef69f05 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:55.443786 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-85702336-e7e9-4852-84ba-fc3aeeb0a7b3 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 25 12:01:55.443786 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:01:55.443786 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:01:55.443786 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:01:55.443786 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:01:55.443786 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:01:55.443786 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:01:55.443786 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:01:55.443786 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:01:55.443786 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:01:55.443786 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:01:55.443786 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:01:55.443786 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 12:01:55.443786 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 12:01:55.443786 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 12:01:55.443786 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 12:01:55.444630 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:01:55.444630 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:01:55.444630 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 12:01:55.444630 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 12:01:55.444630 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 12:01:55.444630 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 12:01:55.444630 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:01:55.444630 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:01:55.444630 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:01:55.444630 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:01:55.444630 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:01:55.444630 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jul 25 12:01:55.444630 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:01:55.444630 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:01:55.444630 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:01:55.444630 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:01:55.444630 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:01:55.444630 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:01:55.445501 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:01:55.445501 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:01:55.445501 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 12:01:55.445501 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 12:01:55.445501 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 12:01:55.445501 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 12:01:55.445501 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 12:01:55.445501 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 12:01:55.445501 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 25 12:01:55.445501 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jul 25 12:01:55.445501 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 251/500] 10.222.0.42 () {56 vars in 1093 bytes} [Mon Jul 25 12:01:55 2022] GET /identity/v3/users/nova => generated 82 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:01:55.446578 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433bdde6-895d-469d-bcb8-0befcd51dc56 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:55.446872 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433bdde6-895d-469d-bcb8-0befcd51dc56 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:55.447152 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433bdde6-895d-469d-bcb8-0befcd51dc56 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:55.448442 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 252/501] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 25 12:01:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:01:55.449128 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433bdde6-895d-469d-bcb8-0befcd51dc56 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:55.450595 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44bbf7f-cca7-43b0-9954-0a382ef69f05 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 12:01:55.458257 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-e44bbf7f-cca7-43b0-9954-0a382ef69f05 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 25 12:01:55.458257 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:01:55.458257 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:01:55.458257 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:01:55.458257 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:01:55.458257 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:01:55.458257 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:01:55.458257 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:01:55.458257 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:01:55.458257 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:01:55.458257 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:01:55.458257 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:01:55.458257 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 12:01:55.458257 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 12:01:55.458257 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 12:01:55.458257 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 12:01:55.459303 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 12:01:55.459303 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 12:01:55.459303 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 12:01:55.459303 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 12:01:55.459303 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:01:55.459303 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:01:55.459303 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:01:55.459303 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:01:55.459303 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:01:55.459303 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:01:55.459303 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:01:55.459303 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 25 12:01:55.459303 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:01:55.459303 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:01:55.459303 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:01:55.459303 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:01:55.460332 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:01:55.460332 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:01:55.460332 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:01:55.460332 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:01:55.460332 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 12:01:55.460332 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 12:01:55.460332 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 12:01:55.460332 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 12:01:55.460332 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 12:01:55.460332 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 12:01:55.460332 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 25 12:01:55.460332 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 25 12:01:55.463686 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 250/502] 10.222.0.42 () {56 vars in 1102 bytes} [Mon Jul 25 12:01:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:01:55.464626 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44bbf7f-cca7-43b0-9954-0a382ef69f05 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:55.512671 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:55.552168 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433bdde6-895d-469d-bcb8-0befcd51dc56 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:55.564704 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44bbf7f-cca7-43b0-9954-0a382ef69f05 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:55.585351 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ca667c-1a50-4dbd-bae2-8792068da214 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:55.586141 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ca667c-1a50-4dbd-bae2-8792068da214 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:55.586433 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ca667c-1a50-4dbd-bae2-8792068da214 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:55.586715 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ca667c-1a50-4dbd-bae2-8792068da214 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:55.597647 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ca667c-1a50-4dbd-bae2-8792068da214 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:55.598663 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02952a30-8e04-4eb3-8122-42ab208406b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:55.599100 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ca667c-1a50-4dbd-bae2-8792068da214 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:55.599329 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02952a30-8e04-4eb3-8122-42ab208406b5 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:55.599644 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02952a30-8e04-4eb3-8122-42ab208406b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:55.599928 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02952a30-8e04-4eb3-8122-42ab208406b5 None admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:55.600776 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02952a30-8e04-4eb3-8122-42ab208406b5 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:01:55.601087 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02952a30-8e04-4eb3-8122-42ab208406b5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:55.602522 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02952a30-8e04-4eb3-8122-42ab208406b5 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:55.616018 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-a9ca667c-1a50-4dbd-bae2-8792068da214 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:01:55.616018 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:01:55.616018 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:01:55.616018 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:01:55.616018 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:01:55.616018 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:01:55.616018 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:01:55.616018 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:01:55.616018 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:01:55.616018 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:01:55.616018 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:01:55.616018 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:01:55.616018 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 12:01:55.616018 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 12:01:55.616018 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 12:01:55.616894 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 12:01:55.616894 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:01:55.616894 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:01:55.616894 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:01:55.616894 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:01:55.616894 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:01:55.616894 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:01:55.616894 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:01:55.616894 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jul 25 12:01:55.616894 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:01:55.616894 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:01:55.616894 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:01:55.616894 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:01:55.616894 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:01:55.616894 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:01:55.616894 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:01:55.616894 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:01:55.617734 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 12:01:55.617734 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 12:01:55.617734 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:01:55.617734 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jul 25 12:01:55.617734 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 253/503] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jul 25 12:01:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:01:55.617987 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ca667c-1a50-4dbd-bae2-8792068da214 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:55.634725 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:55.638990 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 251/504] 10.222.0.42 () {56 vars in 1137 bytes} [Mon Jul 25 12:01:55 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:01:55.640488 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd505d2c-5365-41df-8027-2a48d22e4cf7 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:55.640914 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd505d2c-5365-41df-8027-2a48d22e4cf7 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:55.641200 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd505d2c-5365-41df-8027-2a48d22e4cf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:55.659292 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-cd505d2c-5365-41df-8027-2a48d22e4cf7 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:01:55.660507 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-cd505d2c-5365-41df-8027-2a48d22e4cf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:01:55.723671 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ca667c-1a50-4dbd-bae2-8792068da214 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:55.726969 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 252/505] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 25 12:01:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 12:01:55.727909 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd505d2c-5365-41df-8027-2a48d22e4cf7 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:55.758804 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43794ad-1d60-4496-8edb-125fb40273f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:55.759481 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43794ad-1d60-4496-8edb-125fb40273f4 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:55.759837 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43794ad-1d60-4496-8edb-125fb40273f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:55.760122 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43794ad-1d60-4496-8edb-125fb40273f4 None admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:55.760964 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43794ad-1d60-4496-8edb-125fb40273f4 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:01:55.761291 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43794ad-1d60-4496-8edb-125fb40273f4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:55.762721 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43794ad-1d60-4496-8edb-125fb40273f4 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:55.772836 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:55.800700 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 254/506] 10.222.0.42 () {56 vars in 1133 bytes} [Mon Jul 25 12:01:55 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:01:55.801548 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43794ad-1d60-4496-8edb-125fb40273f4 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:55.839538 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd505d2c-5365-41df-8027-2a48d22e4cf7 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:55.870449 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:55.873228 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4432f07d-39fc-42b6-8e37-e6b73a296983 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:55.873892 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4432f07d-39fc-42b6-8e37-e6b73a296983 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:55.874177 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4432f07d-39fc-42b6-8e37-e6b73a296983 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:55.874473 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4432f07d-39fc-42b6-8e37-e6b73a296983 None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:55.875200 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4432f07d-39fc-42b6-8e37-e6b73a296983 None admin] RBAC: Adding query filter params (name=service) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:01:55.875506 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4432f07d-39fc-42b6-8e37-e6b73a296983 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:55.876923 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4432f07d-39fc-42b6-8e37-e6b73a296983 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:55.885388 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 253/507] 10.222.0.42 () {56 vars in 1103 bytes} [Mon Jul 25 12:01:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:01:55.886102 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4432f07d-39fc-42b6-8e37-e6b73a296983 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:55.903497 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43794ad-1d60-4496-8edb-125fb40273f4 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:55.937655 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b30feaf-d2d4-48ec-9c76-9f552891e47f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:55.938316 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b30feaf-d2d4-48ec-9c76-9f552891e47f None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:55.938602 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b30feaf-d2d4-48ec-9c76-9f552891e47f None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:55.938883 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b30feaf-d2d4-48ec-9c76-9f552891e47f None admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:55.939634 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b30feaf-d2d4-48ec-9c76-9f552891e47f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:01:55.940473 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b30feaf-d2d4-48ec-9c76-9f552891e47f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:55.941854 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b30feaf-d2d4-48ec-9c76-9f552891e47f None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:55.952836 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 255/508] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jul 25 12:01:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:01:55.953608 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b30feaf-d2d4-48ec-9c76-9f552891e47f None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:55.986543 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:55.990800 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4432f07d-39fc-42b6-8e37-e6b73a296983 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:56.024067 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cbf22f-a403-4e28-8d11-f84f1f7a0d42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:56.024789 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cbf22f-a403-4e28-8d11-f84f1f7a0d42 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:56.025148 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cbf22f-a403-4e28-8d11-f84f1f7a0d42 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:56.025430 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cbf22f-a403-4e28-8d11-f84f1f7a0d42 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:56.053997 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b30feaf-d2d4-48ec-9c76-9f552891e47f None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:56.082727 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:56.088906 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69e5866-9733-4272-bcee-650a19f22853 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:56.089737 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69e5866-9733-4272-bcee-650a19f22853 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:56.090022 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69e5866-9733-4272-bcee-650a19f22853 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:56.090299 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69e5866-9733-4272-bcee-650a19f22853 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:56.092292 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69cbf22f-a403-4e28-8d11-f84f1f7a0d42 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:56.093759 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69cbf22f-a403-4e28-8d11-f84f1f7a0d42 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:56.100634 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69e5866-9733-4272-bcee-650a19f22853 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:56.102053 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69e5866-9733-4272-bcee-650a19f22853 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:56.118594 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-f69e5866-9733-4272-bcee-650a19f22853 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:01:56.118594 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:01:56.118594 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:01:56.118594 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:01:56.118594 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:01:56.118594 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:01:56.118594 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:01:56.118594 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:01:56.118594 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:01:56.118594 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:01:56.118594 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:01:56.118594 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:01:56.118594 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 12:01:56.118594 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 12:01:56.118594 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 12:01:56.119434 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 12:01:56.119434 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:01:56.119434 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:01:56.119434 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:01:56.119434 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:01:56.119434 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:01:56.119434 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:01:56.119434 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:01:56.119434 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jul 25 12:01:56.119434 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:01:56.119434 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:01:56.119434 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:01:56.119434 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:01:56.119434 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:01:56.119434 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:01:56.119434 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:01:56.119434 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:01:56.125506 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 12:01:56.125506 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 12:01:56.125506 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:01:56.125506 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jul 25 12:01:56.125506 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 256/509] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jul 25 12:01:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:01:56.125506 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69e5866-9733-4272-bcee-650a19f22853 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:56.200271 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:56.210366 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 254/510] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Jul 25 12:01:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 12:01:56.211240 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cbf22f-a403-4e28-8d11-f84f1f7a0d42 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:56.224355 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69e5866-9733-4272-bcee-650a19f22853 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:56.257796 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef57e1c7-814a-4247-9105-3c71fbbf6a6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:56.258442 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef57e1c7-814a-4247-9105-3c71fbbf6a6e None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:56.258735 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef57e1c7-814a-4247-9105-3c71fbbf6a6e None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:56.259019 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef57e1c7-814a-4247-9105-3c71fbbf6a6e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:56.307940 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:56.308101 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cbf22f-a403-4e28-8d11-f84f1f7a0d42 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:56.326795 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef57e1c7-814a-4247-9105-3c71fbbf6a6e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:56.328293 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef57e1c7-814a-4247-9105-3c71fbbf6a6e None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:56.341541 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9022555-b678-4a36-a629-f29b3debbaa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:56.342322 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9022555-b678-4a36-a629-f29b3debbaa9 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:56.342606 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9022555-b678-4a36-a629-f29b3debbaa9 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:56.342886 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9022555-b678-4a36-a629-f29b3debbaa9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:56.353379 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9022555-b678-4a36-a629-f29b3debbaa9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:56.354824 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9022555-b678-4a36-a629-f29b3debbaa9 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:56.371998 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-f9022555-b678-4a36-a629-f29b3debbaa9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:01:56.371998 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:01:56.371998 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:01:56.371998 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:01:56.371998 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:01:56.371998 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:01:56.371998 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:01:56.371998 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:01:56.371998 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:01:56.371998 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:01:56.371998 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:01:56.371998 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:01:56.371998 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 12:01:56.371998 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 12:01:56.371998 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 12:01:56.372890 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 12:01:56.372890 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:01:56.372890 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:01:56.372890 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:01:56.372890 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:01:56.372890 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:01:56.372890 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:01:56.372890 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:01:56.372890 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 25 12:01:56.372890 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:01:56.372890 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:01:56.372890 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:01:56.372890 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:01:56.372890 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:01:56.372890 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:01:56.372890 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:01:56.372890 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:01:56.373729 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 12:01:56.373729 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 12:01:56.373729 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:01:56.373729 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 25 12:01:56.373729 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 255/511] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jul 25 12:01:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:01:56.373998 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9022555-b678-4a36-a629-f29b3debbaa9 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:56.424270 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:56.434371 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 257/512] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Jul 25 12:01:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 12:01:56.435036 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef57e1c7-814a-4247-9105-3c71fbbf6a6e None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:56.484803 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9022555-b678-4a36-a629-f29b3debbaa9 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:56.516769 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:56.520475 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3d7e09-336d-4290-83b2-11228afea446 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:56.521460 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3d7e09-336d-4290-83b2-11228afea446 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:56.521591 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3d7e09-336d-4290-83b2-11228afea446 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:56.521821 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3d7e09-336d-4290-83b2-11228afea446 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:56.529346 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3d7e09-336d-4290-83b2-11228afea446 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:56.530801 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3d7e09-336d-4290-83b2-11228afea446 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:56.536781 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef57e1c7-814a-4247-9105-3c71fbbf6a6e None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:56.545606 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-da3d7e09-336d-4290-83b2-11228afea446 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 25 12:01:56.545606 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:01:56.545606 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:01:56.545606 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:01:56.545606 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:01:56.545606 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:01:56.545606 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:01:56.545606 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:01:56.545606 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:01:56.545606 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:01:56.545606 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:01:56.545606 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:01:56.545606 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 12:01:56.545606 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 12:01:56.545606 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 12:01:56.545606 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 12:01:56.546543 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:01:56.546543 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:01:56.546543 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 12:01:56.546543 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 12:01:56.546543 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 12:01:56.546543 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 12:01:56.546543 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:01:56.546543 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:01:56.546543 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:01:56.546543 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:01:56.546543 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:01:56.546543 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 25 12:01:56.546543 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:01:56.546543 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:01:56.546543 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:01:56.546543 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:01:56.546543 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:01:56.546543 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:01:56.547423 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:01:56.547423 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:01:56.547423 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 12:01:56.547423 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 12:01:56.547423 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 12:01:56.547423 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 12:01:56.547423 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 12:01:56.547423 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 12:01:56.547423 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 25 12:01:56.547423 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 25 12:01:56.547423 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 256/513] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Jul 25 12:01:56 2022] GET /identity/v3/users/cinder => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:01:56.548008 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3d7e09-336d-4290-83b2-11228afea446 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:56.575693 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2187cd00-45b3-4253-905f-47c8c241907e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:56.576356 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2187cd00-45b3-4253-905f-47c8c241907e None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:56.576645 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2187cd00-45b3-4253-905f-47c8c241907e None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:56.576921 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2187cd00-45b3-4253-905f-47c8c241907e None admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:56.577632 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2187cd00-45b3-4253-905f-47c8c241907e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:01:56.577942 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2187cd00-45b3-4253-905f-47c8c241907e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:56.579292 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2187cd00-45b3-4253-905f-47c8c241907e None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:56.590909 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 258/514] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jul 25 12:01:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:01:56.591854 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2187cd00-45b3-4253-905f-47c8c241907e None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:56.646622 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3d7e09-336d-4290-83b2-11228afea446 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:56.656679 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:56.682184 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f5dade-144a-4821-a874-52c2a92e4550 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:56.682975 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f5dade-144a-4821-a874-52c2a92e4550 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:56.683259 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f5dade-144a-4821-a874-52c2a92e4550 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:56.683559 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f5dade-144a-4821-a874-52c2a92e4550 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:56.691457 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2187cd00-45b3-4253-905f-47c8c241907e None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:56.694621 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f5dade-144a-4821-a874-52c2a92e4550 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:56.696109 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f5dade-144a-4821-a874-52c2a92e4550 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:56.713279 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-86f5dade-144a-4821-a874-52c2a92e4550 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:01:56.713279 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:01:56.713279 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:01:56.713279 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:01:56.713279 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:01:56.713279 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:01:56.713279 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:01:56.713279 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:01:56.713279 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:01:56.713279 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:01:56.713279 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:01:56.713279 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:01:56.713279 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 12:01:56.713279 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 12:01:56.713279 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 12:01:56.714121 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 12:01:56.714121 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:01:56.714121 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:01:56.714121 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:01:56.714121 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:01:56.714121 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:01:56.714121 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:01:56.714121 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:01:56.714121 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 25 12:01:56.714121 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:01:56.714121 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:01:56.714121 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:01:56.714121 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:01:56.714121 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:01:56.714121 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:01:56.714121 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:01:56.714121 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:01:56.714972 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 12:01:56.714972 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 12:01:56.714972 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:01:56.714972 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 25 12:01:56.714972 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 257/515] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jul 25 12:01:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:01:56.715226 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f5dade-144a-4821-a874-52c2a92e4550 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:56.729919 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4391ce-104c-42fd-9c58-d32df6f0eece None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:56.730583 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4391ce-104c-42fd-9c58-d32df6f0eece None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:56.730876 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4391ce-104c-42fd-9c58-d32df6f0eece None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:56.731157 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4391ce-104c-42fd-9c58-d32df6f0eece None admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:56.731913 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4391ce-104c-42fd-9c58-d32df6f0eece None admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:01:56.732228 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4391ce-104c-42fd-9c58-d32df6f0eece None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:56.733585 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4391ce-104c-42fd-9c58-d32df6f0eece None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:56.745622 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 259/516] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jul 25 12:01:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:01:56.746331 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4391ce-104c-42fd-9c58-d32df6f0eece None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:56.748887 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:56.833054 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f5dade-144a-4821-a874-52c2a92e4550 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:56.852259 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4391ce-104c-42fd-9c58-d32df6f0eece None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:56.865130 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b01c47d-a682-4f2e-ae3a-f22437ae793d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:56.865922 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b01c47d-a682-4f2e-ae3a-f22437ae793d None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:56.866209 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b01c47d-a682-4f2e-ae3a-f22437ae793d None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:56.866488 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b01c47d-a682-4f2e-ae3a-f22437ae793d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:56.876829 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b01c47d-a682-4f2e-ae3a-f22437ae793d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:56.878806 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b01c47d-a682-4f2e-ae3a-f22437ae793d None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:56.885802 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c764c691-e072-48fb-96cc-e94685dc60d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:56.886473 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c764c691-e072-48fb-96cc-e94685dc60d9 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:56.886764 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c764c691-e072-48fb-96cc-e94685dc60d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:56.887053 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c764c691-e072-48fb-96cc-e94685dc60d9 None admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:56.887913 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c764c691-e072-48fb-96cc-e94685dc60d9 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:01:56.888236 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c764c691-e072-48fb-96cc-e94685dc60d9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:56.889657 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c764c691-e072-48fb-96cc-e94685dc60d9 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:56.896618 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-6b01c47d-a682-4f2e-ae3a-f22437ae793d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:01:56.896618 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:01:56.896618 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:01:56.896618 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:01:56.896618 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:01:56.896618 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:01:56.896618 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:01:56.896618 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:01:56.896618 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:01:56.896618 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:01:56.896618 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:01:56.896618 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:01:56.896618 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 12:01:56.896618 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 12:01:56.896618 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 12:01:56.897463 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 12:01:56.897463 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:01:56.897463 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:01:56.897463 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:01:56.897463 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:01:56.897463 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:01:56.897463 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:01:56.897463 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:01:56.897463 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 25 12:01:56.897463 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:01:56.897463 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:01:56.897463 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:01:56.897463 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:01:56.897463 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:01:56.897463 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:01:56.897463 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:01:56.897463 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:01:56.898317 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 12:01:56.898317 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 12:01:56.898317 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:01:56.898317 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 25 12:01:56.898317 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 258/517] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jul 25 12:01:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:01:56.898580 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b01c47d-a682-4f2e-ae3a-f22437ae793d None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:56.908862 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 260/518] 10.222.0.42 () {56 vars in 1137 bytes} [Mon Jul 25 12:01:56 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:01:56.909740 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c764c691-e072-48fb-96cc-e94685dc60d9 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:56.916138 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:56.969676 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:57.001060 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b01c47d-a682-4f2e-ae3a-f22437ae793d None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:57.010326 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c764c691-e072-48fb-96cc-e94685dc60d9 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:57.034976 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e161af-572c-4c2c-a4c7-d17f5ebad5fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:57.035681 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e161af-572c-4c2c-a4c7-d17f5ebad5fc None admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:57.035970 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e161af-572c-4c2c-a4c7-d17f5ebad5fc None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:57.036249 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e161af-572c-4c2c-a4c7-d17f5ebad5fc None admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:57.036955 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e161af-572c-4c2c-a4c7-d17f5ebad5fc None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:57.038433 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e161af-572c-4c2c-a4c7-d17f5ebad5fc None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:57.044264 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9a1de1-0e39-4a43-bb75-7b3a4818a6e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:57.044925 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9a1de1-0e39-4a43-bb75-7b3a4818a6e9 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:57.045214 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9a1de1-0e39-4a43-bb75-7b3a4818a6e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:57.045498 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9a1de1-0e39-4a43-bb75-7b3a4818a6e9 None admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:57.046230 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9a1de1-0e39-4a43-bb75-7b3a4818a6e9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:01:57.046701 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9a1de1-0e39-4a43-bb75-7b3a4818a6e9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:57.048111 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9a1de1-0e39-4a43-bb75-7b3a4818a6e9 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:57.060392 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 259/519] 10.222.0.42 () {60 vars in 1132 bytes} [Mon Jul 25 12:01:56 2022] POST /identity/v3/users => generated 247 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:01:57.060563 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 261/520] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jul 25 12:01:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:01:57.061067 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9a1de1-0e39-4a43-bb75-7b3a4818a6e9 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:57.064501 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e161af-572c-4c2c-a4c7-d17f5ebad5fc None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:57.144846 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:57.180471 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9a1de1-0e39-4a43-bb75-7b3a4818a6e9 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:57.181833 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e161af-572c-4c2c-a4c7-d17f5ebad5fc None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:57.204139 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:57.223067 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a181eb4-fcf3-4064-8a2e-e910ed3d41e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:57.223637 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d728a86-0132-4ab9-884a-4de6bdf7bf14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:57.224242 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a181eb4-fcf3-4064-8a2e-e910ed3d41e4 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:57.224444 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d728a86-0132-4ab9-884a-4de6bdf7bf14 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:57.224736 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d728a86-0132-4ab9-884a-4de6bdf7bf14 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:57.224863 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a181eb4-fcf3-4064-8a2e-e910ed3d41e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:57.225591 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d728a86-0132-4ab9-884a-4de6bdf7bf14 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:57.225696 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a181eb4-fcf3-4064-8a2e-e910ed3d41e4 None admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:57.225868 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a181eb4-fcf3-4064-8a2e-e910ed3d41e4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:01:57.226174 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a181eb4-fcf3-4064-8a2e-e910ed3d41e4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:57.227704 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a181eb4-fcf3-4064-8a2e-e910ed3d41e4 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:57.234218 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d728a86-0132-4ab9-884a-4de6bdf7bf14 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:57.235720 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d728a86-0132-4ab9-884a-4de6bdf7bf14 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:57.242616 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 262/521] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jul 25 12:01:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:01:57.243861 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a181eb4-fcf3-4064-8a2e-e910ed3d41e4 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:57.251352 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-1d728a86-0132-4ab9-884a-4de6bdf7bf14 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 25 12:01:57.251352 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:01:57.251352 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:01:57.251352 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:01:57.251352 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:01:57.251352 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:01:57.251352 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:01:57.251352 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:01:57.251352 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:01:57.251352 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:01:57.251352 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:01:57.251352 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:01:57.251352 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 12:01:57.251352 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 12:01:57.251352 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 12:01:57.251352 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 12:01:57.252972 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:01:57.252972 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:01:57.252972 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 12:01:57.252972 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 12:01:57.252972 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 12:01:57.252972 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 12:01:57.252972 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:01:57.252972 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:01:57.252972 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:01:57.252972 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:01:57.252972 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:01:57.252972 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 25 12:01:57.252972 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:01:57.252972 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:01:57.252972 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:01:57.252972 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:01:57.252972 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:01:57.252972 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:01:57.253864 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:01:57.253864 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:01:57.253864 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 12:01:57.253864 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 12:01:57.253864 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 12:01:57.253864 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 12:01:57.253864 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 12:01:57.253864 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 12:01:57.253864 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 25 12:01:57.253864 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 25 12:01:57.253864 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 260/522] 10.222.0.42 () {56 vars in 1102 bytes} [Mon Jul 25 12:01:57 2022] GET /identity/v3/users/neutron => generated 85 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:01:57.254976 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d728a86-0132-4ab9-884a-4de6bdf7bf14 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:57.367123 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a181eb4-fcf3-4064-8a2e-e910ed3d41e4 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:57.371346 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d728a86-0132-4ab9-884a-4de6bdf7bf14 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:57.400592 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:57.405851 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8d8b81-a925-4672-9799-e9604cde9fa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:57.406522 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8d8b81-a925-4672-9799-e9604cde9fa3 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:57.406814 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8d8b81-a925-4672-9799-e9604cde9fa3 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:57.407100 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8d8b81-a925-4672-9799-e9604cde9fa3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:57.408150 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358a57b3-7559-4a16-aa28-5b69c2a5f181 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:57.409616 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358a57b3-7559-4a16-aa28-5b69c2a5f181 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:57.410047 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358a57b3-7559-4a16-aa28-5b69c2a5f181 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:57.410630 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358a57b3-7559-4a16-aa28-5b69c2a5f181 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:57.418678 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358a57b3-7559-4a16-aa28-5b69c2a5f181 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:57.420254 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358a57b3-7559-4a16-aa28-5b69c2a5f181 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:57.437317 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-358a57b3-7559-4a16-aa28-5b69c2a5f181 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 12:01:57.437317 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:01:57.437317 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:01:57.437317 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:01:57.437317 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:01:57.437317 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:01:57.437317 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:01:57.437317 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:01:57.437317 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:01:57.437317 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:01:57.437317 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:01:57.437317 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:01:57.437317 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 12:01:57.437317 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 12:01:57.437317 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 12:01:57.438203 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 12:01:57.438203 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:01:57.438203 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:01:57.438203 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:01:57.438203 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:01:57.438203 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:01:57.438203 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:01:57.438203 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:01:57.438203 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 25 12:01:57.438203 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:01:57.438203 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:01:57.438203 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:01:57.438203 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:01:57.438203 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:01:57.438203 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:01:57.438203 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:01:57.438203 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:01:57.439179 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 12:01:57.439179 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 12:01:57.439179 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 12:01:57.439179 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 12:01:57.439179 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 12:01:57.439179 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 12:01:57.439179 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 12:01:57.439179 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 25 12:01:57.439179 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 261/523] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Jul 25 12:01:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:01:57.439731 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358a57b3-7559-4a16-aa28-5b69c2a5f181 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:57.464677 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:57.492113 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8d8b81-a925-4672-9799-e9604cde9fa3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:57.493577 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8d8b81-a925-4672-9799-e9604cde9fa3 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:57.555056 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358a57b3-7559-4a16-aa28-5b69c2a5f181 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:57.593825 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec4f5de-117b-42e5-a10a-449873e3be0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:57.594948 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec4f5de-117b-42e5-a10a-449873e3be0b None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:57.595509 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec4f5de-117b-42e5-a10a-449873e3be0b None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:57.595901 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec4f5de-117b-42e5-a10a-449873e3be0b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:57.603495 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 263/524] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Jul 25 12:01:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 12:01:57.604416 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8d8b81-a925-4672-9799-e9604cde9fa3 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:57.608853 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec4f5de-117b-42e5-a10a-449873e3be0b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:57.610322 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec4f5de-117b-42e5-a10a-449873e3be0b None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:57.624896 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-dec4f5de-117b-42e5-a10a-449873e3be0b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 12:01:57.624896 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:01:57.624896 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:01:57.624896 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:01:57.624896 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:01:57.624896 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:01:57.624896 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:01:57.624896 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:01:57.624896 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:01:57.624896 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:01:57.624896 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:01:57.624896 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:01:57.624896 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 12:01:57.624896 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 12:01:57.624896 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 12:01:57.625792 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 12:01:57.625792 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:01:57.625792 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:01:57.625792 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:01:57.625792 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:01:57.625792 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:01:57.625792 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:01:57.625792 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:01:57.625792 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 25 12:01:57.625792 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:01:57.625792 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:01:57.625792 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:01:57.625792 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:01:57.625792 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:01:57.625792 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:01:57.625792 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:01:57.625792 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:01:57.626638 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 12:01:57.626638 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 12:01:57.626638 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 12:01:57.626638 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 12:01:57.626638 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 12:01:57.626638 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 12:01:57.626638 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 12:01:57.626638 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 25 12:01:57.626638 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 262/525] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Jul 25 12:01:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:01:57.628751 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec4f5de-117b-42e5-a10a-449873e3be0b None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:57.681425 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:57.693113 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:57.715157 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8d8b81-a925-4672-9799-e9604cde9fa3 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:57.738691 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec4f5de-117b-42e5-a10a-449873e3be0b None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:57.752884 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30127f6e-5415-4c54-8c76-d5d31c1639a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:57.753561 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30127f6e-5415-4c54-8c76-d5d31c1639a6 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:57.753848 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30127f6e-5415-4c54-8c76-d5d31c1639a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:57.754126 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30127f6e-5415-4c54-8c76-d5d31c1639a6 None admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:57.754971 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30127f6e-5415-4c54-8c76-d5d31c1639a6 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:01:57.755290 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30127f6e-5415-4c54-8c76-d5d31c1639a6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:57.756774 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30127f6e-5415-4c54-8c76-d5d31c1639a6 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:57.779686 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8790a4-ff05-4b6c-8d72-50b0f42ffe00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:57.780132 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8790a4-ff05-4b6c-8d72-50b0f42ffe00 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:57.780665 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 264/526] 10.222.0.42 () {56 vars in 1139 bytes} [Mon Jul 25 12:01:57 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:01:57.780773 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8790a4-ff05-4b6c-8d72-50b0f42ffe00 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:57.781002 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8790a4-ff05-4b6c-8d72-50b0f42ffe00 None admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:57.781584 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30127f6e-5415-4c54-8c76-d5d31c1639a6 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:57.781823 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8790a4-ff05-4b6c-8d72-50b0f42ffe00 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:01:57.782144 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8790a4-ff05-4b6c-8d72-50b0f42ffe00 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:57.783666 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8790a4-ff05-4b6c-8d72-50b0f42ffe00 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:57.803077 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 263/527] 10.222.0.42 () {56 vars in 1148 bytes} [Mon Jul 25 12:01:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:01:57.804318 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8790a4-ff05-4b6c-8d72-50b0f42ffe00 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:57.892150 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30127f6e-5415-4c54-8c76-d5d31c1639a6 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:57.905025 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8790a4-ff05-4b6c-8d72-50b0f42ffe00 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:57.917344 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:57.938176 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3fb053-ae43-49ba-878e-b976ce1a7c75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:57.938976 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3fb053-ae43-49ba-878e-b976ce1a7c75 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:57.939267 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3fb053-ae43-49ba-878e-b976ce1a7c75 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:57.939567 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3fb053-ae43-49ba-878e-b976ce1a7c75 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:57.941864 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8670786e-c544-47d7-9239-0912ff178545 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:57.942522 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8670786e-c544-47d7-9239-0912ff178545 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:57.942807 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8670786e-c544-47d7-9239-0912ff178545 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:57.943087 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8670786e-c544-47d7-9239-0912ff178545 None admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:57.943902 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8670786e-c544-47d7-9239-0912ff178545 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:01:57.944215 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8670786e-c544-47d7-9239-0912ff178545 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:57.945648 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8670786e-c544-47d7-9239-0912ff178545 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:57.946345 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:57.951721 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3fb053-ae43-49ba-878e-b976ce1a7c75 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:57.953302 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3fb053-ae43-49ba-878e-b976ce1a7c75 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:57.963152 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 264/528] 10.222.0.42 () {56 vars in 1148 bytes} [Mon Jul 25 12:01:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:01:57.963152 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8670786e-c544-47d7-9239-0912ff178545 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:57.973357 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-dd3fb053-ae43-49ba-878e-b976ce1a7c75 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:01:57.973357 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:01:57.973357 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:01:57.973357 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:01:57.973357 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:01:57.973357 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:01:57.973357 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:01:57.973357 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:01:57.973357 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:01:57.973357 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:01:57.973357 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:01:57.973357 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:01:57.973357 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 12:01:57.973357 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 12:01:57.973357 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 12:01:57.974209 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 12:01:57.974209 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:01:57.974209 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:01:57.974209 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:01:57.974209 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:01:57.974209 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:01:57.974209 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:01:57.974209 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:01:57.974209 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jul 25 12:01:57.974209 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:01:57.974209 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:01:57.974209 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:01:57.974209 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:01:57.974209 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:01:57.974209 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:01:57.974209 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:01:57.974209 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:01:57.975024 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 12:01:57.975024 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 12:01:57.975024 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:01:57.975024 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jul 25 12:01:57.979029 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 265/529] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jul 25 12:01:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:01:57.979713 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3fb053-ae43-49ba-878e-b976ce1a7c75 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:58.067262 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8670786e-c544-47d7-9239-0912ff178545 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:58.100916 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3fb053-ae43-49ba-878e-b976ce1a7c75 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:58.103555 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3118dc72-1621-4f06-8f2a-6ca14dbcdbb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:58.104313 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3118dc72-1621-4f06-8f2a-6ca14dbcdbb5 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:58.104607 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3118dc72-1621-4f06-8f2a-6ca14dbcdbb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:58.104889 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3118dc72-1621-4f06-8f2a-6ca14dbcdbb5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:58.136344 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce7ec19-c88a-4afc-9388-f639faeed039 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:58.137297 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce7ec19-c88a-4afc-9388-f639faeed039 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:58.137653 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce7ec19-c88a-4afc-9388-f639faeed039 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:58.137930 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce7ec19-c88a-4afc-9388-f639faeed039 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:58.138756 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce7ec19-c88a-4afc-9388-f639faeed039 None admin] RBAC: Adding query filter params (role.id=79dec36aee5b4322bf652041c25ca6e4, scope.project.id=b9f62b8f37304b8a9a4bb54205347577, user.id=5ea16528b1374bae9cf54603f7907bfd) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:01:58.139072 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce7ec19-c88a-4afc-9388-f639faeed039 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:58.140531 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce7ec19-c88a-4afc-9388-f639faeed039 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:58.151649 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 266/530] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Jul 25 12:01:57 2022] GET /identity/v3/role_assignments?user.id=5ea16528b1374bae9cf54603f7907bfd&scope.project.id=b9f62b8f37304b8a9a4bb54205347577&role.id=79dec36aee5b4322bf652041c25ca6e4 => generated 259 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:01:58.151649 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce7ec19-c88a-4afc-9388-f639faeed039 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:58.160189 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:58.195814 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3118dc72-1621-4f06-8f2a-6ca14dbcdbb5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:58.197595 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3118dc72-1621-4f06-8f2a-6ca14dbcdbb5 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:58.225260 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:58.275297 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce7ec19-c88a-4afc-9388-f639faeed039 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:58.312092 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 265/531] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Jul 25 12:01:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 12:01:58.313043 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3118dc72-1621-4f06-8f2a-6ca14dbcdbb5 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:58.318071 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886f20de-6810-4743-8e24-c67a77bab586 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:58.318749 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886f20de-6810-4743-8e24-c67a77bab586 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:58.319039 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886f20de-6810-4743-8e24-c67a77bab586 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:58.319325 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886f20de-6810-4743-8e24-c67a77bab586 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:58.320292 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886f20de-6810-4743-8e24-c67a77bab586 None admin] RBAC: Adding query filter params (role.id=79dec36aee5b4322bf652041c25ca6e4, scope.project.id=b9f62b8f37304b8a9a4bb54205347577, user.id=f67142a1193d47c7bd33d7040268233c) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:01:58.320614 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886f20de-6810-4743-8e24-c67a77bab586 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:58.322239 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886f20de-6810-4743-8e24-c67a77bab586 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:58.328808 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 267/532] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Jul 25 12:01:58 2022] GET /identity/v3/role_assignments?user.id=f67142a1193d47c7bd33d7040268233c&scope.project.id=b9f62b8f37304b8a9a4bb54205347577&role.id=79dec36aee5b4322bf652041c25ca6e4 => generated 259 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:01:58.329827 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886f20de-6810-4743-8e24-c67a77bab586 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:58.391730 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:58.423828 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3118dc72-1621-4f06-8f2a-6ca14dbcdbb5 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:58.449934 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886f20de-6810-4743-8e24-c67a77bab586 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:58.469229 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:58.488255 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5170a3dc-aea7-4b55-ba7f-aeaad4018efe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:58.489082 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5170a3dc-aea7-4b55-ba7f-aeaad4018efe None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:58.489781 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5170a3dc-aea7-4b55-ba7f-aeaad4018efe None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:58.490540 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5170a3dc-aea7-4b55-ba7f-aeaad4018efe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:58.502047 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84aeac0-8a46-4e21-8d4e-7020f9fa3d09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:58.503673 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84aeac0-8a46-4e21-8d4e-7020f9fa3d09 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:58.503995 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5170a3dc-aea7-4b55-ba7f-aeaad4018efe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:58.504352 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84aeac0-8a46-4e21-8d4e-7020f9fa3d09 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:58.504794 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84aeac0-8a46-4e21-8d4e-7020f9fa3d09 None admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:58.505704 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84aeac0-8a46-4e21-8d4e-7020f9fa3d09 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:01:58.505821 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5170a3dc-aea7-4b55-ba7f-aeaad4018efe None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:58.506292 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84aeac0-8a46-4e21-8d4e-7020f9fa3d09 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:58.508062 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84aeac0-8a46-4e21-8d4e-7020f9fa3d09 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:58.526531 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-5170a3dc-aea7-4b55-ba7f-aeaad4018efe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:01:58.526531 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:01:58.526531 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:01:58.526531 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:01:58.526531 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:01:58.526531 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:01:58.526531 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:01:58.526531 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:01:58.526531 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:01:58.526531 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:01:58.526531 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:01:58.526531 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:01:58.526531 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 12:01:58.526531 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 12:01:58.526531 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 12:01:58.527432 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 12:01:58.527432 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:01:58.527432 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:01:58.527432 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:01:58.527432 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:01:58.527432 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:01:58.527432 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:01:58.527432 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:01:58.527432 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jul 25 12:01:58.527432 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:01:58.527432 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:01:58.527432 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:01:58.527432 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:01:58.527432 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:01:58.527432 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:01:58.527432 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:01:58.527432 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:01:58.528313 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 12:01:58.528313 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 12:01:58.528313 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:01:58.528313 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jul 25 12:01:58.528528 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 268/533] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jul 25 12:01:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:01:58.533853 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5170a3dc-aea7-4b55-ba7f-aeaad4018efe None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:58.536600 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 266/534] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jul 25 12:01:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:01:58.543730 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84aeac0-8a46-4e21-8d4e-7020f9fa3d09 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:58.659468 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5170a3dc-aea7-4b55-ba7f-aeaad4018efe None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:58.697622 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4176ae70-4337-4ac1-bb17-71b77209894a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:58.698549 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4176ae70-4337-4ac1-bb17-71b77209894a None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:58.699113 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4176ae70-4337-4ac1-bb17-71b77209894a None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:58.699467 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4176ae70-4337-4ac1-bb17-71b77209894a None admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:58.701247 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4176ae70-4337-4ac1-bb17-71b77209894a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:01:58.701655 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4176ae70-4337-4ac1-bb17-71b77209894a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:58.703131 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4176ae70-4337-4ac1-bb17-71b77209894a None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:58.718743 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 269/535] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jul 25 12:01:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 12:01:58.724503 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4176ae70-4337-4ac1-bb17-71b77209894a None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:58.745464 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84aeac0-8a46-4e21-8d4e-7020f9fa3d09 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:58.792723 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:58.809979 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:58.834572 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275debe6-d556-45dd-b894-49df482b80cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:58.836821 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275debe6-d556-45dd-b894-49df482b80cc None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:58.837700 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275debe6-d556-45dd-b894-49df482b80cc None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:58.838939 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275debe6-d556-45dd-b894-49df482b80cc None admin] PATH_INFO: `/v3/projects/service` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:58.854745 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275debe6-d556-45dd-b894-49df482b80cc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:58.856704 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275debe6-d556-45dd-b894-49df482b80cc None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:58.886793 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-275debe6-d556-45dd-b894-49df482b80cc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 12:01:58.886793 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:01:58.886793 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:01:58.886793 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:01:58.886793 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:01:58.886793 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:01:58.886793 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:01:58.886793 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:01:58.886793 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:01:58.886793 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:01:58.886793 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:01:58.886793 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:01:58.886793 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 12:01:58.886793 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 12:01:58.886793 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 12:01:58.887735 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 12:01:58.887735 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:01:58.887735 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:01:58.887735 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:01:58.887735 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:01:58.887735 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:01:58.887735 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:01:58.887735 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:01:58.887735 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 25 12:01:58.887735 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:01:58.887735 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:01:58.887735 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:01:58.887735 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:01:58.887735 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:01:58.887735 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:01:58.887735 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:01:58.887735 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:01:58.888566 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 12:01:58.888566 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 12:01:58.888566 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 12:01:58.888566 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 12:01:58.888566 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 12:01:58.888566 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 12:01:58.888566 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 12:01:58.888566 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 25 12:01:58.895484 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 267/536] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Jul 25 12:01:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 351 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 12:01:58.900334 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275debe6-d556-45dd-b894-49df482b80cc None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:58.939389 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4176ae70-4337-4ac1-bb17-71b77209894a None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:59.009953 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7af0a6-761d-47fd-8399-3639b5d5b03d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:59.011188 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7af0a6-761d-47fd-8399-3639b5d5b03d None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:59.011715 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7af0a6-761d-47fd-8399-3639b5d5b03d None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:59.012196 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7af0a6-761d-47fd-8399-3639b5d5b03d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:59.031020 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7af0a6-761d-47fd-8399-3639b5d5b03d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:59.032523 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7af0a6-761d-47fd-8399-3639b5d5b03d None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:59.050983 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275debe6-d556-45dd-b894-49df482b80cc None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:59.068924 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-2b7af0a6-761d-47fd-8399-3639b5d5b03d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 12:01:59.068924 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:01:59.068924 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:01:59.068924 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:01:59.068924 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:01:59.068924 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:01:59.068924 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:01:59.068924 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:01:59.068924 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:01:59.068924 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:01:59.068924 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:01:59.068924 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:01:59.068924 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 12:01:59.068924 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 12:01:59.068924 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 12:01:59.069784 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 12:01:59.069784 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:01:59.069784 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:01:59.069784 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:01:59.069784 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:01:59.069784 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:01:59.069784 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:01:59.069784 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:01:59.069784 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jul 25 12:01:59.069784 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:01:59.069784 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:01:59.069784 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:01:59.069784 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:01:59.069784 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:01:59.069784 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:01:59.069784 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:01:59.069784 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:01:59.070650 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 12:01:59.070650 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 12:01:59.070650 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 12:01:59.070650 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 12:01:59.070650 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 12:01:59.070650 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 12:01:59.070650 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 12:01:59.070650 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jul 25 12:01:59.070650 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 270/537] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Jul 25 12:01:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 346 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:01:59.076209 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7af0a6-761d-47fd-8399-3639b5d5b03d None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:59.140997 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e750849b-53ba-48a8-9ef4-3838142aeaa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:59.142053 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e750849b-53ba-48a8-9ef4-3838142aeaa1 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:59.142563 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e750849b-53ba-48a8-9ef4-3838142aeaa1 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:59.144080 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e750849b-53ba-48a8-9ef4-3838142aeaa1 None admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:59.146223 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e750849b-53ba-48a8-9ef4-3838142aeaa1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:01:59.146775 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e750849b-53ba-48a8-9ef4-3838142aeaa1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:59.151203 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e750849b-53ba-48a8-9ef4-3838142aeaa1 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:59.177811 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 268/538] 10.222.0.42 () {56 vars in 1148 bytes} [Mon Jul 25 12:01:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:01:59.180888 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e750849b-53ba-48a8-9ef4-3838142aeaa1 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:59.225392 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:59.235912 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7af0a6-761d-47fd-8399-3639b5d5b03d None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:59.257946 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:59.274550 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4461e0-77e1-4274-8f1f-307b5dc1331a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:59.275177 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4461e0-77e1-4274-8f1f-307b5dc1331a None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:59.275533 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4461e0-77e1-4274-8f1f-307b5dc1331a None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:59.275843 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4461e0-77e1-4274-8f1f-307b5dc1331a None admin] PATH_INFO: `/v3/projects` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:59.276881 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4461e0-77e1-4274-8f1f-307b5dc1331a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:01:59.277077 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4461e0-77e1-4274-8f1f-307b5dc1331a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:59.278472 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4461e0-77e1-4274-8f1f-307b5dc1331a None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:59.298520 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 271/539] 10.222.0.42 () {56 vars in 1148 bytes} [Mon Jul 25 12:01:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:01:59.300498 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178e59f8-1468-49e8-b4fc-96868d2a259b None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:59.300791 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178e59f8-1468-49e8-b4fc-96868d2a259b None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:59.301233 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178e59f8-1468-49e8-b4fc-96868d2a259b None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:59.306641 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 272/540] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 25 12:01:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:01:59.307411 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178e59f8-1468-49e8-b4fc-96868d2a259b None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:59.327652 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e750849b-53ba-48a8-9ef4-3838142aeaa1 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:59.363795 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e5a70f-0066-4178-8cef-7347d28d8469 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:59.364583 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e5a70f-0066-4178-8cef-7347d28d8469 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:59.364794 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e5a70f-0066-4178-8cef-7347d28d8469 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:59.365072 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e5a70f-0066-4178-8cef-7347d28d8469 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:59.365966 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e5a70f-0066-4178-8cef-7347d28d8469 None admin] RBAC: Adding query filter params (role.id=79dec36aee5b4322bf652041c25ca6e4, scope.project.id=b9f62b8f37304b8a9a4bb54205347577, user.id=6f89ce6f0cf04373a414d927fa0f8a7f) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:01:59.366281 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e5a70f-0066-4178-8cef-7347d28d8469 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:59.367814 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e5a70f-0066-4178-8cef-7347d28d8469 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:59.374042 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 269/541] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Jul 25 12:01:59 2022] GET /identity/v3/role_assignments?user.id=6f89ce6f0cf04373a414d927fa0f8a7f&scope.project.id=b9f62b8f37304b8a9a4bb54205347577&role.id=79dec36aee5b4322bf652041c25ca6e4 => generated 259 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:01:59.375424 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a893c4-6024-459b-8f51-018b55a653ed None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:59.375743 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a893c4-6024-459b-8f51-018b55a653ed None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:59.376021 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a893c4-6024-459b-8f51-018b55a653ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:59.396417 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-88a893c4-6024-459b-8f51-018b55a653ed None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:01:59.397636 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-88a893c4-6024-459b-8f51-018b55a653ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:01:59.447218 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178e59f8-1468-49e8-b4fc-96868d2a259b None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:59.474829 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 270/542] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 25 12:01:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 12:01:59.490052 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e29b2a-c6aa-4506-8571-2e4093772bba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:59.491116 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e29b2a-c6aa-4506-8571-2e4093772bba None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:59.491644 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e29b2a-c6aa-4506-8571-2e4093772bba None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:59.492129 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e29b2a-c6aa-4506-8571-2e4093772bba None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:59.493196 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e29b2a-c6aa-4506-8571-2e4093772bba None admin] RBAC: Adding query filter params (role.id=79dec36aee5b4322bf652041c25ca6e4, scope.project.id=b9f62b8f37304b8a9a4bb54205347577, user.id=51733781201243e8abf9eca39b207750) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:01:59.493748 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e29b2a-c6aa-4506-8571-2e4093772bba None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:59.495459 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e29b2a-c6aa-4506-8571-2e4093772bba None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:59.499956 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a893c4-6024-459b-8f51-018b55a653ed None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:59.503488 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 273/543] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Jul 25 12:01:59 2022] GET /identity/v3/role_assignments?user.id=51733781201243e8abf9eca39b207750&scope.project.id=b9f62b8f37304b8a9a4bb54205347577&role.id=79dec36aee5b4322bf652041c25ca6e4 => generated 259 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:01:59.557650 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:59.620534 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a893c4-6024-459b-8f51-018b55a653ed None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:59.659062 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde9cf35-0309-4191-9086-b964afe2d885 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:59.659906 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde9cf35-0309-4191-9086-b964afe2d885 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:59.660191 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde9cf35-0309-4191-9086-b964afe2d885 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:59.660466 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde9cf35-0309-4191-9086-b964afe2d885 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:59.672569 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde9cf35-0309-4191-9086-b964afe2d885 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 12:01:59.681308 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-fde9cf35-0309-4191-9086-b964afe2d885 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 25 12:01:59.681308 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:01:59.681308 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:01:59.681308 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:01:59.681308 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:01:59.681308 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:01:59.681308 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:01:59.681308 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:01:59.681308 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:01:59.681308 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:01:59.681308 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:01:59.681308 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:01:59.681308 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 12:01:59.681308 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 12:01:59.681308 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 12:01:59.681308 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 12:01:59.682251 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 12:01:59.682251 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 12:01:59.682251 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 12:01:59.682251 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 12:01:59.682251 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:01:59.682251 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:01:59.682251 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:01:59.682251 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:01:59.682251 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:01:59.682251 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:01:59.682251 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:01:59.682251 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 25 12:01:59.682251 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:01:59.682251 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:01:59.682251 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:01:59.682251 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:01:59.683050 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:01:59.683050 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:01:59.683050 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:01:59.683050 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:01:59.683050 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 12:01:59.683050 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 12:01:59.683050 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 12:01:59.683050 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 12:01:59.683050 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 12:01:59.683050 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 12:01:59.683050 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 25 12:01:59.683050 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 25 12:01:59.687651 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 271/544] 10.222.0.42 () {56 vars in 1102 bytes} [Mon Jul 25 12:01:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:01:59.689088 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e29b2a-c6aa-4506-8571-2e4093772bba None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:01:59.707015 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:01:59.830107 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e29b2a-c6aa-4506-8571-2e4093772bba None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:01:59.877902 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3f400b-7c62-4537-b65c-ff9ab79e316f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:01:59.878585 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3f400b-7c62-4537-b65c-ff9ab79e316f None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:01:59.878871 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3f400b-7c62-4537-b65c-ff9ab79e316f None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:01:59.879155 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3f400b-7c62-4537-b65c-ff9ab79e316f None admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:01:59.879938 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3f400b-7c62-4537-b65c-ff9ab79e316f None admin] RBAC: Adding query filter params (name=service) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:01:59.880252 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3f400b-7c62-4537-b65c-ff9ab79e316f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:01:59.881624 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3f400b-7c62-4537-b65c-ff9ab79e316f None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:01:59.895210 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 274/545] 10.222.0.42 () {56 vars in 1103 bytes} [Mon Jul 25 12:01:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:01:59.900742 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde9cf35-0309-4191-9086-b964afe2d885 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:00.021773 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde9cf35-0309-4191-9086-b964afe2d885 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:00.032537 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:00.057513 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579bdd3f-5665-4c6e-9d6f-7c63859ca19d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:00.058314 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579bdd3f-5665-4c6e-9d6f-7c63859ca19d None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:00.058656 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579bdd3f-5665-4c6e-9d6f-7c63859ca19d None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:00.058945 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579bdd3f-5665-4c6e-9d6f-7c63859ca19d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:00.070083 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579bdd3f-5665-4c6e-9d6f-7c63859ca19d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:00.071568 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579bdd3f-5665-4c6e-9d6f-7c63859ca19d None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:00.089831 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-579bdd3f-5665-4c6e-9d6f-7c63859ca19d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:02:00.089831 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:02:00.089831 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:02:00.089831 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:02:00.089831 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:02:00.089831 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:02:00.089831 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:02:00.089831 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:02:00.089831 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:02:00.089831 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:02:00.089831 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:02:00.089831 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:02:00.089831 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 12:02:00.089831 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 12:02:00.089831 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 12:02:00.090692 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 12:02:00.090692 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:02:00.090692 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:02:00.090692 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:02:00.090692 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:02:00.090692 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:02:00.090692 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:02:00.090692 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:02:00.090692 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 25 12:02:00.090692 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:02:00.090692 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:02:00.090692 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:02:00.090692 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:02:00.090692 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:02:00.090692 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:02:00.090692 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:02:00.090692 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:02:00.091533 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 12:02:00.091533 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 12:02:00.091533 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:02:00.091533 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 25 12:02:00.095536 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 272/546] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jul 25 12:01:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:02:00.097455 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3f400b-7c62-4537-b65c-ff9ab79e316f None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:00.184632 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:00.208631 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3f400b-7c62-4537-b65c-ff9ab79e316f None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:00.253037 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cc01c3-b8ce-4738-a51a-2bf6ff5a926b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:00.253724 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cc01c3-b8ce-4738-a51a-2bf6ff5a926b None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:00.254015 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cc01c3-b8ce-4738-a51a-2bf6ff5a926b None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:00.254299 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cc01c3-b8ce-4738-a51a-2bf6ff5a926b None admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:00.255033 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14cc01c3-b8ce-4738-a51a-2bf6ff5a926b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:02:00.255349 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14cc01c3-b8ce-4738-a51a-2bf6ff5a926b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:00.256777 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14cc01c3-b8ce-4738-a51a-2bf6ff5a926b None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:00.279031 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 275/547] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jul 25 12:02:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 12:02:00.284445 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579bdd3f-5665-4c6e-9d6f-7c63859ca19d None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:00.409556 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579bdd3f-5665-4c6e-9d6f-7c63859ca19d None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:00.444724 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd952cda-b868-4eac-8e3d-8701f245d4b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:00.445531 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd952cda-b868-4eac-8e3d-8701f245d4b1 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:00.445818 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd952cda-b868-4eac-8e3d-8701f245d4b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:00.446097 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd952cda-b868-4eac-8e3d-8701f245d4b1 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:00.454124 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd952cda-b868-4eac-8e3d-8701f245d4b1 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:00.455649 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd952cda-b868-4eac-8e3d-8701f245d4b1 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:00.468952 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0fab5a-5628-41a9-b1ba-25aeaf5a8a78 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:00.469243 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0fab5a-5628-41a9-b1ba-25aeaf5a8a78 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:00.469527 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0fab5a-5628-41a9-b1ba-25aeaf5a8a78 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:00.470053 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-bd952cda-b868-4eac-8e3d-8701f245d4b1 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 25 12:02:00.470053 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:02:00.470053 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:02:00.470053 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:02:00.470053 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:02:00.470053 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:02:00.470053 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:02:00.470053 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:02:00.470053 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:02:00.470053 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:02:00.470053 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:02:00.470053 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:02:00.470053 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 12:02:00.470053 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 12:02:00.470053 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 12:02:00.470053 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 12:02:00.471455 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:02:00.471455 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:02:00.471455 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 12:02:00.471455 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 12:02:00.471455 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 12:02:00.471455 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 12:02:00.471455 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:02:00.471455 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:02:00.471455 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:02:00.471455 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:02:00.471455 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:02:00.471455 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 25 12:02:00.471455 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:02:00.471455 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:02:00.471455 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:02:00.471455 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:02:00.471455 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:02:00.471455 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:02:00.474060 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:02:00.474060 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:02:00.474060 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 12:02:00.474060 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 12:02:00.474060 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 12:02:00.474060 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 12:02:00.474060 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 12:02:00.474060 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 12:02:00.474060 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 25 12:02:00.474060 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 25 12:02:00.484288 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 276/548] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 25 12:02:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 12:02:00.484442 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 273/549] 10.222.0.42 () {56 vars in 1096 bytes} [Mon Jul 25 12:02:00 2022] GET /identity/v3/users/swift => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:02:00.484584 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8952fea-d16d-48a5-8d28-abb0f51cb87f None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:00.484584 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8952fea-d16d-48a5-8d28-abb0f51cb87f None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:00.484584 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8952fea-d16d-48a5-8d28-abb0f51cb87f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:00.485044 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd952cda-b868-4eac-8e3d-8701f245d4b1 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:00.498560 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:00.524593 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-c8952fea-d16d-48a5-8d28-abb0f51cb87f None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:02:00.526171 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-c8952fea-d16d-48a5-8d28-abb0f51cb87f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:02:00.588197 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd952cda-b868-4eac-8e3d-8701f245d4b1 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:00.627606 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8d338d-84e9-40e3-b531-9b1119f844fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:00.627606 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8d338d-84e9-40e3-b531-9b1119f844fc None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:00.627606 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8d338d-84e9-40e3-b531-9b1119f844fc None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:00.627606 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8d338d-84e9-40e3-b531-9b1119f844fc None admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:00.628015 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8d338d-84e9-40e3-b531-9b1119f844fc None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:02:00.628015 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8d338d-84e9-40e3-b531-9b1119f844fc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:00.628015 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8d338d-84e9-40e3-b531-9b1119f844fc None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:00.649021 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 277/550] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 25 12:02:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 12:02:00.663680 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 274/551] 10.222.0.42 () {56 vars in 1135 bytes} [Mon Jul 25 12:02:00 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 12:02:00.665017 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8952fea-d16d-48a5-8d28-abb0f51cb87f None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:00.676676 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8d338d-84e9-40e3-b531-9b1119f844fc None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:00.759388 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:00.802131 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8952fea-d16d-48a5-8d28-abb0f51cb87f None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:00.840083 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379c4224-ca8b-4bb3-a5e0-e0965d70dec5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:00.840747 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379c4224-ca8b-4bb3-a5e0-e0965d70dec5 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:00.841085 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379c4224-ca8b-4bb3-a5e0-e0965d70dec5 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:00.841367 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379c4224-ca8b-4bb3-a5e0-e0965d70dec5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:00.897920 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:00.922280 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379c4224-ca8b-4bb3-a5e0-e0965d70dec5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:00.924530 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379c4224-ca8b-4bb3-a5e0-e0965d70dec5 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:00.956238 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8d338d-84e9-40e3-b531-9b1119f844fc None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:01.007459 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:01.011748 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da4f93a-b2d8-4fa6-82dc-a0251eb68dc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:01.012570 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da4f93a-b2d8-4fa6-82dc-a0251eb68dc8 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:01.012883 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da4f93a-b2d8-4fa6-82dc-a0251eb68dc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:01.013166 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da4f93a-b2d8-4fa6-82dc-a0251eb68dc8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:01.024553 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da4f93a-b2d8-4fa6-82dc-a0251eb68dc8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 12:02:01.032577 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-0da4f93a-b2d8-4fa6-82dc-a0251eb68dc8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 25 12:02:01.032577 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:02:01.032577 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:02:01.032577 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:02:01.032577 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:02:01.032577 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:02:01.032577 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:02:01.032577 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:02:01.032577 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:02:01.032577 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:02:01.032577 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:02:01.032577 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:02:01.032577 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 12:02:01.032577 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 12:02:01.032577 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 12:02:01.032577 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 12:02:01.034111 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 12:02:01.034111 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 12:02:01.034111 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 12:02:01.034111 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 12:02:01.034111 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:02:01.034111 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:02:01.034111 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:02:01.034111 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:02:01.034111 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:02:01.034111 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:02:01.034111 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:02:01.034111 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 25 12:02:01.034111 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:02:01.034111 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:02:01.034111 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:02:01.034111 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:02:01.035013 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:02:01.035013 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:02:01.035013 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:02:01.035013 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:02:01.035013 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 12:02:01.035013 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 12:02:01.035013 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 12:02:01.035013 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 12:02:01.035013 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 12:02:01.035013 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 12:02:01.035013 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 25 12:02:01.035013 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 25 12:02:01.039239 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 275/552] 10.222.0.42 () {56 vars in 1102 bytes} [Mon Jul 25 12:02:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 363 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:02:01.040601 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcc50c9-9f7f-4d91-b2f8-5ed009caa939 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:01.041147 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcc50c9-9f7f-4d91-b2f8-5ed009caa939 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:01.041433 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcc50c9-9f7f-4d91-b2f8-5ed009caa939 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:01.042964 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 276/553] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 25 12:02:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:02:01.047589 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcc50c9-9f7f-4d91-b2f8-5ed009caa939 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:01.051634 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 278/554] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Jul 25 12:02:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 25 12:02:01.053885 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe89edd-d90f-4b3b-aeec-f74146b537ce None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:01.054175 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe89edd-d90f-4b3b-aeec-f74146b537ce None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:01.054463 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe89edd-d90f-4b3b-aeec-f74146b537ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:01.072581 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-dbe89edd-d90f-4b3b-aeec-f74146b537ce None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:02:01.073949 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-dbe89edd-d90f-4b3b-aeec-f74146b537ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:02:01.161858 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 279/555] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 25 12:02:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 12:02:01.163530 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe89edd-d90f-4b3b-aeec-f74146b537ce None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:01.177182 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcc50c9-9f7f-4d91-b2f8-5ed009caa939 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:01.228727 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd32ad27-a46e-4b88-8169-0cefb4906b58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:01.229729 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd32ad27-a46e-4b88-8169-0cefb4906b58 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:01.230331 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd32ad27-a46e-4b88-8169-0cefb4906b58 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:01.230770 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd32ad27-a46e-4b88-8169-0cefb4906b58 None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:01.232153 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd32ad27-a46e-4b88-8169-0cefb4906b58 None admin] RBAC: Adding query filter params (name=service) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:02:01.232735 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd32ad27-a46e-4b88-8169-0cefb4906b58 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:01.234718 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd32ad27-a46e-4b88-8169-0cefb4906b58 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:01.236514 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:01.247960 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 277/556] 10.222.0.42 () {56 vars in 1103 bytes} [Mon Jul 25 12:02:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:02:01.249029 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd32ad27-a46e-4b88-8169-0cefb4906b58 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:01.274311 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:01.282150 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe89edd-d90f-4b3b-aeec-f74146b537ce None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:01.328648 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9731ce5d-1f8c-4a7a-853f-25d0d88a2cf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:01.330046 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9731ce5d-1f8c-4a7a-853f-25d0d88a2cf8 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:01.330899 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9731ce5d-1f8c-4a7a-853f-25d0d88a2cf8 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:01.331182 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9731ce5d-1f8c-4a7a-853f-25d0d88a2cf8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:01.342073 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9731ce5d-1f8c-4a7a-853f-25d0d88a2cf8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:01.343534 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9731ce5d-1f8c-4a7a-853f-25d0d88a2cf8 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:01.361642 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-9731ce5d-1f8c-4a7a-853f-25d0d88a2cf8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:02:01.361642 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:02:01.361642 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:02:01.361642 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:02:01.361642 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:02:01.361642 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:02:01.361642 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:02:01.361642 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:02:01.361642 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:02:01.361642 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:02:01.361642 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:02:01.361642 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:02:01.361642 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 12:02:01.361642 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 12:02:01.361642 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 12:02:01.362560 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 12:02:01.362560 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:02:01.362560 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:02:01.362560 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:02:01.362560 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:02:01.362560 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:02:01.362560 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:02:01.362560 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:02:01.362560 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jul 25 12:02:01.362560 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:02:01.362560 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:02:01.362560 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:02:01.362560 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:02:01.362560 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:02:01.362560 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:02:01.362560 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:02:01.362560 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:02:01.363402 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 12:02:01.363402 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 12:02:01.363402 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:02:01.363402 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jul 25 12:02:01.367497 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 280/557] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jul 25 12:02:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:02:01.368552 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9731ce5d-1f8c-4a7a-853f-25d0d88a2cf8 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:01.379159 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd32ad27-a46e-4b88-8169-0cefb4906b58 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:01.423675 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06d4c79-9278-4dd3-bd43-79d5e87d89d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:01.425297 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06d4c79-9278-4dd3-bd43-79d5e87d89d3 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:01.426033 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06d4c79-9278-4dd3-bd43-79d5e87d89d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:01.426516 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06d4c79-9278-4dd3-bd43-79d5e87d89d3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:01.441251 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06d4c79-9278-4dd3-bd43-79d5e87d89d3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 12:02:01.451362 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-b06d4c79-9278-4dd3-bd43-79d5e87d89d3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 25 12:02:01.451362 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:02:01.451362 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:02:01.451362 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:02:01.451362 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:02:01.451362 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:02:01.451362 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:02:01.451362 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:02:01.451362 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:02:01.451362 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:02:01.451362 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:02:01.451362 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:02:01.451362 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 12:02:01.451362 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 12:02:01.451362 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 12:02:01.451362 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 12:02:01.452850 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 12:02:01.452850 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 12:02:01.452850 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 12:02:01.452850 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 12:02:01.452850 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:02:01.452850 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:02:01.452850 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:02:01.452850 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:02:01.452850 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:02:01.452850 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:02:01.452850 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:02:01.452850 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 25 12:02:01.452850 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:02:01.452850 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:02:01.452850 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:02:01.452850 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:02:01.453677 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:02:01.453677 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:02:01.453677 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:02:01.453677 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:02:01.453677 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 12:02:01.453677 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 12:02:01.453677 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 12:02:01.453677 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 12:02:01.453677 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 12:02:01.453677 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 12:02:01.453677 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 25 12:02:01.453677 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 25 12:02:01.456994 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 278/558] 10.222.0.42 () {56 vars in 1102 bytes} [Mon Jul 25 12:02:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:02:01.458023 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06d4c79-9278-4dd3-bd43-79d5e87d89d3 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:01.482550 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9731ce5d-1f8c-4a7a-853f-25d0d88a2cf8 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:01.524998 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc5a6df-b61c-4ca6-8b7a-d3017ab971bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:01.526630 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc5a6df-b61c-4ca6-8b7a-d3017ab971bd None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:01.527098 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc5a6df-b61c-4ca6-8b7a-d3017ab971bd None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:01.527829 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc5a6df-b61c-4ca6-8b7a-d3017ab971bd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:01.530310 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:01.543919 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc5a6df-b61c-4ca6-8b7a-d3017ab971bd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:01.545899 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc5a6df-b61c-4ca6-8b7a-d3017ab971bd None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:01.557989 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:01.571038 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-cbc5a6df-b61c-4ca6-8b7a-d3017ab971bd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:02:01.571038 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:02:01.571038 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:02:01.571038 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:02:01.571038 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:02:01.571038 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:02:01.571038 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:02:01.571038 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:02:01.571038 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:02:01.571038 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:02:01.571038 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:02:01.571038 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:02:01.571038 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 12:02:01.571038 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 12:02:01.571038 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 12:02:01.572041 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 12:02:01.572041 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:02:01.572041 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:02:01.572041 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:02:01.572041 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:02:01.572041 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:02:01.572041 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:02:01.572041 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:02:01.572041 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jul 25 12:02:01.572041 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:02:01.572041 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:02:01.572041 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:02:01.572041 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:02:01.572041 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:02:01.572041 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:02:01.572041 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:02:01.572041 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:02:01.573011 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 12:02:01.573011 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 12:02:01.573011 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:02:01.573011 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jul 25 12:02:01.575340 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 281/559] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jul 25 12:02:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:02:01.576338 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc5a6df-b61c-4ca6-8b7a-d3017ab971bd None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:01.593766 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06d4c79-9278-4dd3-bd43-79d5e87d89d3 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:01.640561 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261d5eba-cc9c-4091-acc1-57215d2a0013 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:01.641515 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261d5eba-cc9c-4091-acc1-57215d2a0013 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:01.642134 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261d5eba-cc9c-4091-acc1-57215d2a0013 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:01.642601 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261d5eba-cc9c-4091-acc1-57215d2a0013 None admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:01.644456 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261d5eba-cc9c-4091-acc1-57215d2a0013 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:02:01.645187 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261d5eba-cc9c-4091-acc1-57215d2a0013 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:01.646915 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261d5eba-cc9c-4091-acc1-57215d2a0013 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:01.664072 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 279/560] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jul 25 12:02:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:02:01.665215 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261d5eba-cc9c-4091-acc1-57215d2a0013 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:01.708896 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc5a6df-b61c-4ca6-8b7a-d3017ab971bd None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:01.745990 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ab10a9-876c-4acf-8c94-e5f1087b5ca1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:01.746852 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ab10a9-876c-4acf-8c94-e5f1087b5ca1 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:01.747273 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ab10a9-876c-4acf-8c94-e5f1087b5ca1 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:01.747702 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ab10a9-876c-4acf-8c94-e5f1087b5ca1 None admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:01.748564 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ab10a9-876c-4acf-8c94-e5f1087b5ca1 None admin] RBAC: Adding query filter params (name=service) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:02:01.748977 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ab10a9-876c-4acf-8c94-e5f1087b5ca1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:01.750537 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ab10a9-876c-4acf-8c94-e5f1087b5ca1 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:01.764298 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 282/561] 10.222.0.42 () {56 vars in 1103 bytes} [Mon Jul 25 12:02:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:02:01.765295 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ab10a9-876c-4acf-8c94-e5f1087b5ca1 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:01.795172 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261d5eba-cc9c-4091-acc1-57215d2a0013 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:01.814727 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:01.834792 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a4a0f6-910e-447d-887f-6adccc0aa9ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:01.835389 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a4a0f6-910e-447d-887f-6adccc0aa9ea None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:01.835720 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a4a0f6-910e-447d-887f-6adccc0aa9ea None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:01.836000 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a4a0f6-910e-447d-887f-6adccc0aa9ea None admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:01.836726 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a4a0f6-910e-447d-887f-6adccc0aa9ea None admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:02:01.837034 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a4a0f6-910e-447d-887f-6adccc0aa9ea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:01.838404 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a4a0f6-910e-447d-887f-6adccc0aa9ea None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:01.850688 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:01.857328 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 280/562] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jul 25 12:02:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:02:01.858296 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a4a0f6-910e-447d-887f-6adccc0aa9ea None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:01.899346 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ab10a9-876c-4acf-8c94-e5f1087b5ca1 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:01.932791 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768dad07-5fa9-4cf6-87ff-7536e9d5df83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:01.933585 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768dad07-5fa9-4cf6-87ff-7536e9d5df83 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:01.933873 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768dad07-5fa9-4cf6-87ff-7536e9d5df83 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:01.934155 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768dad07-5fa9-4cf6-87ff-7536e9d5df83 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:01.941437 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768dad07-5fa9-4cf6-87ff-7536e9d5df83 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:01.942869 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768dad07-5fa9-4cf6-87ff-7536e9d5df83 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:01.957420 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-768dad07-5fa9-4cf6-87ff-7536e9d5df83 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 12:02:01.957420 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:02:01.957420 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:02:01.957420 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:02:01.957420 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:02:01.957420 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:02:01.957420 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:02:01.957420 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:02:01.957420 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:02:01.957420 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:02:01.957420 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:02:01.957420 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:02:01.957420 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 12:02:01.957420 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 12:02:01.957420 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 12:02:01.958455 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 12:02:01.958455 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:02:01.958455 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:02:01.958455 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:02:01.958455 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:02:01.958455 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:02:01.958455 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:02:01.958455 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:02:01.958455 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jul 25 12:02:01.958455 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:02:01.958455 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:02:01.958455 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:02:01.958455 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:02:01.958455 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:02:01.958455 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:02:01.958455 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:02:01.958455 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:02:01.959380 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 12:02:01.959380 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 12:02:01.959380 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 12:02:01.959380 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 12:02:01.959380 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 12:02:01.959380 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 12:02:01.959380 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 12:02:01.959380 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jul 25 12:02:01.959380 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 283/563] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Jul 25 12:02:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:02:01.959929 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768dad07-5fa9-4cf6-87ff-7536e9d5df83 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:01.967936 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a4a0f6-910e-447d-887f-6adccc0aa9ea None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:02.000785 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae8d617-dcb9-47c5-9ba3-0e2035ce9c70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:02.001580 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae8d617-dcb9-47c5-9ba3-0e2035ce9c70 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:02.001870 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae8d617-dcb9-47c5-9ba3-0e2035ce9c70 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:02.002673 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae8d617-dcb9-47c5-9ba3-0e2035ce9c70 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:02.013136 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae8d617-dcb9-47c5-9ba3-0e2035ce9c70 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:02.014578 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae8d617-dcb9-47c5-9ba3-0e2035ce9c70 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:02.031988 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-5ae8d617-dcb9-47c5-9ba3-0e2035ce9c70 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:02:02.031988 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:02:02.031988 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:02:02.031988 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:02:02.031988 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:02:02.031988 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:02:02.031988 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:02:02.031988 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:02:02.031988 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:02:02.031988 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:02:02.031988 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:02:02.031988 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:02:02.031988 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 12:02:02.031988 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 12:02:02.031988 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 12:02:02.032902 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 12:02:02.032902 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:02:02.032902 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:02:02.032902 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:02:02.032902 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:02:02.032902 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:02:02.032902 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:02:02.032902 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:02:02.032902 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 25 12:02:02.032902 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:02:02.032902 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:02:02.032902 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:02:02.032902 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:02:02.032902 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:02:02.032902 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:02:02.032902 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:02:02.032902 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:02:02.033883 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 12:02:02.033883 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 12:02:02.033883 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:02:02.033883 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 25 12:02:02.033883 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 281/564] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jul 25 12:02:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:02:02.034596 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4d1b2f-4e1e-4458-8927-1e23757891bd None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:02.034721 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4d1b2f-4e1e-4458-8927-1e23757891bd None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:02.034923 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4d1b2f-4e1e-4458-8927-1e23757891bd None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:02.036192 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 282/565] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 25 12:02:02 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:02:02.037246 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c32913-313c-4286-b041-eeaa32c5c267 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:02.037532 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c32913-313c-4286-b041-eeaa32c5c267 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:02.037811 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c32913-313c-4286-b041-eeaa32c5c267 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:02.039008 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 283/566] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 25 12:02:02 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:02:02.039795 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c32913-313c-4286-b041-eeaa32c5c267 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:02.064464 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768dad07-5fa9-4cf6-87ff-7536e9d5df83 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:02.069503 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:02.088125 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:02.099801 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4e3976-75f0-4dae-b483-465008511280 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:02.100588 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4e3976-75f0-4dae-b483-465008511280 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:02.100871 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4e3976-75f0-4dae-b483-465008511280 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:02.101149 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4e3976-75f0-4dae-b483-465008511280 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:02.108039 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4e3976-75f0-4dae-b483-465008511280 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:02.109461 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4e3976-75f0-4dae-b483-465008511280 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:02.123380 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-5f4e3976-75f0-4dae-b483-465008511280 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 25 12:02:02.123380 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:02:02.123380 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:02:02.123380 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:02:02.123380 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:02:02.123380 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:02:02.123380 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:02:02.123380 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:02:02.123380 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:02:02.123380 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:02:02.123380 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:02:02.123380 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:02:02.123380 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 12:02:02.123380 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 12:02:02.123380 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 12:02:02.123380 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 12:02:02.124556 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:02:02.124556 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:02:02.124556 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 12:02:02.124556 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 12:02:02.124556 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 12:02:02.124556 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 12:02:02.124556 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:02:02.124556 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:02:02.124556 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:02:02.124556 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:02:02.124556 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:02:02.124556 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jul 25 12:02:02.124556 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:02:02.124556 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:02:02.124556 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:02:02.124556 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:02:02.124556 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:02:02.124556 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:02:02.125636 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:02:02.125636 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:02:02.125636 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 12:02:02.125636 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 12:02:02.125636 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 12:02:02.125636 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 12:02:02.125636 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 12:02:02.125636 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 12:02:02.125636 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 25 12:02:02.125636 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jul 25 12:02:02.125636 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 284/567] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Jul 25 12:02:01 2022] GET /identity/v3/users/glance => generated 84 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:02:02.125636 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4e3976-75f0-4dae-b483-465008511280 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:02.147641 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c32913-313c-4286-b041-eeaa32c5c267 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:02.180614 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33293b1-b142-4f40-b80e-1434e6fe3521 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:02.181895 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33293b1-b142-4f40-b80e-1434e6fe3521 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:02.182183 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33293b1-b142-4f40-b80e-1434e6fe3521 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:02.182469 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33293b1-b142-4f40-b80e-1434e6fe3521 None admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:02.183266 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33293b1-b142-4f40-b80e-1434e6fe3521 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:02:02.183610 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33293b1-b142-4f40-b80e-1434e6fe3521 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:02.185047 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33293b1-b142-4f40-b80e-1434e6fe3521 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:02.197358 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 284/568] 10.222.0.42 () {56 vars in 1148 bytes} [Mon Jul 25 12:02:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:02:02.198529 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919b3bdf-fb44-4016-9bc5-7065ddc78fab None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:02.198825 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919b3bdf-fb44-4016-9bc5-7065ddc78fab None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:02.199107 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919b3bdf-fb44-4016-9bc5-7065ddc78fab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:02.216923 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-919b3bdf-fb44-4016-9bc5-7065ddc78fab None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:02:02.218037 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-919b3bdf-fb44-4016-9bc5-7065ddc78fab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:02:02.227257 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4e3976-75f0-4dae-b483-465008511280 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:02.260241 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d281960-fefe-4a36-8113-1f5d42ca5034 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:02.260894 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d281960-fefe-4a36-8113-1f5d42ca5034 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:02.261180 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d281960-fefe-4a36-8113-1f5d42ca5034 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:02.261456 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d281960-fefe-4a36-8113-1f5d42ca5034 None admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:02.262171 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d281960-fefe-4a36-8113-1f5d42ca5034 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:02:02.262475 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d281960-fefe-4a36-8113-1f5d42ca5034 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:02.263871 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d281960-fefe-4a36-8113-1f5d42ca5034 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:02.276543 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 285/569] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jul 25 12:02:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:02:02.277807 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1881242c-499e-4908-bb36-226fba3236f9 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:02.278103 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1881242c-499e-4908-bb36-226fba3236f9 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:02.278385 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1881242c-499e-4908-bb36-226fba3236f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:02.288045 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 285/570] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 25 12:02:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 12:02:02.288878 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919b3bdf-fb44-4016-9bc5-7065ddc78fab None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:02.296839 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-1881242c-499e-4908-bb36-226fba3236f9 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:02:02.298141 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-1881242c-499e-4908-bb36-226fba3236f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:02:02.309055 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:02.334763 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:02.367095 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 286/571] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 25 12:02:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 12:02:02.367950 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1881242c-499e-4908-bb36-226fba3236f9 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:02.393748 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919b3bdf-fb44-4016-9bc5-7065ddc78fab None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:02.427692 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05654f64-e9ed-4c72-9ffe-3adf82c56eda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:02.428364 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05654f64-e9ed-4c72-9ffe-3adf82c56eda None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:02.428651 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05654f64-e9ed-4c72-9ffe-3adf82c56eda None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:02.428934 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05654f64-e9ed-4c72-9ffe-3adf82c56eda None admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:02.429770 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05654f64-e9ed-4c72-9ffe-3adf82c56eda None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:02:02.430084 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05654f64-e9ed-4c72-9ffe-3adf82c56eda None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:02.431532 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05654f64-e9ed-4c72-9ffe-3adf82c56eda None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:02.449541 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 286/572] 10.222.0.42 () {56 vars in 1137 bytes} [Mon Jul 25 12:02:02 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:02:02.450434 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05654f64-e9ed-4c72-9ffe-3adf82c56eda None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:02.474120 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1881242c-499e-4908-bb36-226fba3236f9 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:02.507817 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f27432-3dbf-4623-a775-5f3706727c08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:02.508502 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f27432-3dbf-4623-a775-5f3706727c08 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:02.508937 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f27432-3dbf-4623-a775-5f3706727c08 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:02.509266 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f27432-3dbf-4623-a775-5f3706727c08 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:02.510203 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f27432-3dbf-4623-a775-5f3706727c08 None admin] RBAC: Adding query filter params (role.id=79dec36aee5b4322bf652041c25ca6e4, scope.project.id=b9f62b8f37304b8a9a4bb54205347577, user.id=7bd782ea66364eeb96592096eae63d4f) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:02:02.510518 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f27432-3dbf-4623-a775-5f3706727c08 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:02.511964 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f27432-3dbf-4623-a775-5f3706727c08 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:02.517509 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 287/573] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Jul 25 12:02:02 2022] GET /identity/v3/role_assignments?user.id=7bd782ea66364eeb96592096eae63d4f&scope.project.id=b9f62b8f37304b8a9a4bb54205347577&role.id=79dec36aee5b4322bf652041c25ca6e4 => generated 259 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:02:02.518114 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f27432-3dbf-4623-a775-5f3706727c08 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:02.530073 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:02.560072 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:02.561824 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05654f64-e9ed-4c72-9ffe-3adf82c56eda None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:02.599261 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e4d58a-813d-497f-8300-343a3d60593a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:02.600121 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e4d58a-813d-497f-8300-343a3d60593a None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:02.600409 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e4d58a-813d-497f-8300-343a3d60593a None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:02.600699 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e4d58a-813d-497f-8300-343a3d60593a None admin] PATH_INFO: `/v3/users/nova` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:02.609207 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e4d58a-813d-497f-8300-343a3d60593a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:02.610694 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e4d58a-813d-497f-8300-343a3d60593a None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:02.628359 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-c3e4d58a-813d-497f-8300-343a3d60593a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 25 12:02:02.628359 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:02:02.628359 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:02:02.628359 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:02:02.628359 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:02:02.628359 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:02:02.628359 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:02:02.628359 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:02:02.628359 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:02:02.628359 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:02:02.628359 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:02:02.628359 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:02:02.628359 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 12:02:02.628359 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 12:02:02.628359 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 12:02:02.628359 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 12:02:02.629469 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:02:02.629469 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:02:02.629469 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 12:02:02.629469 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 12:02:02.629469 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 12:02:02.629469 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 12:02:02.629469 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:02:02.629469 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:02:02.629469 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:02:02.629469 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:02:02.629469 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:02:02.629469 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 25 12:02:02.629469 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:02:02.629469 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:02:02.629469 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:02:02.629469 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:02:02.629469 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:02:02.629469 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:02:02.630436 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:02:02.630436 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:02:02.630436 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 12:02:02.630436 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 12:02:02.630436 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 12:02:02.630436 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 12:02:02.630436 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 12:02:02.630436 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 12:02:02.630436 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 25 12:02:02.630436 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 25 12:02:02.630961 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 287/574] 10.222.0.42 () {56 vars in 1093 bytes} [Mon Jul 25 12:02:02 2022] GET /identity/v3/users/nova => generated 82 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:02:02.632071 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e4d58a-813d-497f-8300-343a3d60593a None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:02.648091 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f27432-3dbf-4623-a775-5f3706727c08 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:02.690910 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd6b973-7cf2-4136-b69a-edb39e17e610 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:02.691747 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd6b973-7cf2-4136-b69a-edb39e17e610 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:02.692033 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd6b973-7cf2-4136-b69a-edb39e17e610 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:02.692312 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd6b973-7cf2-4136-b69a-edb39e17e610 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:02.703592 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd6b973-7cf2-4136-b69a-edb39e17e610 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 12:02:02.711779 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-6cd6b973-7cf2-4136-b69a-edb39e17e610 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 25 12:02:02.711779 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:02:02.711779 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:02:02.711779 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:02:02.711779 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:02:02.711779 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:02:02.711779 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:02:02.711779 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:02:02.711779 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:02:02.711779 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:02:02.711779 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:02:02.711779 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:02:02.711779 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 12:02:02.711779 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 12:02:02.711779 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 12:02:02.711779 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 12:02:02.712798 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 12:02:02.712798 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 12:02:02.712798 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 12:02:02.712798 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 12:02:02.712798 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:02:02.712798 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:02:02.712798 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:02:02.712798 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:02:02.712798 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:02:02.712798 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:02:02.712798 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:02:02.712798 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jul 25 12:02:02.712798 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:02:02.712798 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:02:02.712798 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:02:02.712798 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:02:02.718944 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:02:02.718944 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:02:02.718944 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:02:02.718944 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:02:02.718944 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 12:02:02.718944 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 12:02:02.718944 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 12:02:02.718944 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 12:02:02.718944 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 12:02:02.718944 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 12:02:02.718944 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 25 12:02:02.718944 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jul 25 12:02:02.718944 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 288/575] 10.222.0.42 () {56 vars in 1102 bytes} [Mon Jul 25 12:02:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:02:02.718944 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd6b973-7cf2-4136-b69a-edb39e17e610 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:02.747409 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e4d58a-813d-497f-8300-343a3d60593a None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:02.788974 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0c2200-f6da-44a1-be29-f1137778d82f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:02.789840 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0c2200-f6da-44a1-be29-f1137778d82f None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:02.790177 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0c2200-f6da-44a1-be29-f1137778d82f None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:02.790458 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0c2200-f6da-44a1-be29-f1137778d82f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:02.793423 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:02.804372 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0c2200-f6da-44a1-be29-f1137778d82f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 12:02:02.812772 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-ee0c2200-f6da-44a1-be29-f1137778d82f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 25 12:02:02.812772 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:02:02.812772 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:02:02.812772 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:02:02.812772 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:02:02.812772 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:02:02.812772 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:02:02.812772 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:02:02.812772 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:02:02.812772 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:02:02.812772 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:02:02.812772 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:02:02.812772 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 12:02:02.812772 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 12:02:02.812772 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 12:02:02.812772 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 12:02:02.813696 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 12:02:02.813696 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 12:02:02.813696 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 12:02:02.813696 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 12:02:02.813696 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:02:02.813696 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:02:02.813696 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:02:02.813696 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:02:02.813696 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:02:02.813696 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:02:02.813696 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:02:02.813696 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 25 12:02:02.813696 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:02:02.813696 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:02:02.813696 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:02:02.813696 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:02:02.814573 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:02:02.814573 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:02:02.814573 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:02:02.814573 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:02:02.814573 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 12:02:02.814573 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 12:02:02.814573 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 12:02:02.814573 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 12:02:02.814573 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 12:02:02.814573 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 12:02:02.814573 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 25 12:02:02.814573 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 25 12:02:02.814573 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 288/576] 10.222.0.42 () {56 vars in 1102 bytes} [Mon Jul 25 12:02:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:02:02.815256 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0c2200-f6da-44a1-be29-f1137778d82f None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:02.828286 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:02.835638 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd6b973-7cf2-4136-b69a-edb39e17e610 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:02.875090 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b273eba-a106-47f2-81de-d9464f630b32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:02.876169 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b273eba-a106-47f2-81de-d9464f630b32 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:02.876461 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b273eba-a106-47f2-81de-d9464f630b32 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:02.876894 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b273eba-a106-47f2-81de-d9464f630b32 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:02.888986 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b273eba-a106-47f2-81de-d9464f630b32 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:02.890526 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b273eba-a106-47f2-81de-d9464f630b32 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:02.908328 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-6b273eba-a106-47f2-81de-d9464f630b32 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:02:02.908328 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:02:02.908328 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:02:02.908328 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:02:02.908328 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:02:02.908328 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:02:02.908328 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:02:02.908328 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:02:02.908328 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:02:02.908328 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:02:02.908328 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:02:02.908328 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:02:02.908328 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 12:02:02.908328 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 12:02:02.908328 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 12:02:02.909423 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 12:02:02.909423 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:02:02.909423 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:02:02.909423 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:02:02.909423 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:02:02.909423 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:02:02.909423 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:02:02.909423 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:02:02.909423 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jul 25 12:02:02.909423 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:02:02.909423 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:02:02.909423 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:02:02.909423 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:02:02.909423 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:02:02.909423 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:02:02.909423 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:02:02.909423 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:02:02.910321 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 12:02:02.910321 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 12:02:02.910321 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:02:02.910321 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jul 25 12:02:02.910321 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 289/577] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jul 25 12:02:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:02:02.910321 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b273eba-a106-47f2-81de-d9464f630b32 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:02.927173 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0c2200-f6da-44a1-be29-f1137778d82f None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:02.962573 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf3f700-a78c-4ccc-95d0-f391adb67c55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:02.963255 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf3f700-a78c-4ccc-95d0-f391adb67c55 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:02.963554 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf3f700-a78c-4ccc-95d0-f391adb67c55 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:02.963847 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf3f700-a78c-4ccc-95d0-f391adb67c55 None admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:02.964720 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf3f700-a78c-4ccc-95d0-f391adb67c55 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:02:02.965028 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf3f700-a78c-4ccc-95d0-f391adb67c55 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:02.966459 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf3f700-a78c-4ccc-95d0-f391adb67c55 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:02.987600 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 289/578] 10.222.0.42 () {56 vars in 1133 bytes} [Mon Jul 25 12:02:02 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:02:02.987600 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf3f700-a78c-4ccc-95d0-f391adb67c55 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:03.030619 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b273eba-a106-47f2-81de-d9464f630b32 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:03.040216 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:03.069741 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c485c8f-0d4e-4762-8f16-a0461d74371d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:03.070526 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c485c8f-0d4e-4762-8f16-a0461d74371d None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:03.070934 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c485c8f-0d4e-4762-8f16-a0461d74371d None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:03.071219 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c485c8f-0d4e-4762-8f16-a0461d74371d None admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:03.071999 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c485c8f-0d4e-4762-8f16-a0461d74371d None admin] RBAC: Adding query filter params (name=service) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:02:03.072317 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c485c8f-0d4e-4762-8f16-a0461d74371d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:03.073807 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c485c8f-0d4e-4762-8f16-a0461d74371d None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:03.082583 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 290/579] 10.222.0.42 () {56 vars in 1103 bytes} [Mon Jul 25 12:02:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:02:03.083364 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c485c8f-0d4e-4762-8f16-a0461d74371d None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:03.090177 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf3f700-a78c-4ccc-95d0-f391adb67c55 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:03.095658 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:03.125699 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdc69f3-1d2f-4cc6-abf0-6cb78a0b560d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:03.126373 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdc69f3-1d2f-4cc6-abf0-6cb78a0b560d None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:03.126662 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdc69f3-1d2f-4cc6-abf0-6cb78a0b560d None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:03.126942 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdc69f3-1d2f-4cc6-abf0-6cb78a0b560d None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:03.128274 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fdc69f3-1d2f-4cc6-abf0-6cb78a0b560d None admin] RBAC: Adding query filter params (name=service) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:02:03.128583 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fdc69f3-1d2f-4cc6-abf0-6cb78a0b560d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:03.129953 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fdc69f3-1d2f-4cc6-abf0-6cb78a0b560d None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:03.138507 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 290/580] 10.222.0.42 () {56 vars in 1103 bytes} [Mon Jul 25 12:02:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:02:03.139306 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdc69f3-1d2f-4cc6-abf0-6cb78a0b560d None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:03.198451 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c485c8f-0d4e-4762-8f16-a0461d74371d None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:03.232826 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78feb4e1-b107-4bad-bcc0-ede98883684c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:03.233907 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78feb4e1-b107-4bad-bcc0-ede98883684c None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:03.234333 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78feb4e1-b107-4bad-bcc0-ede98883684c None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:03.234766 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78feb4e1-b107-4bad-bcc0-ede98883684c None admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:03.235670 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78feb4e1-b107-4bad-bcc0-ede98883684c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:02:03.235993 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78feb4e1-b107-4bad-bcc0-ede98883684c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:03.238188 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78feb4e1-b107-4bad-bcc0-ede98883684c None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:03.245494 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdc69f3-1d2f-4cc6-abf0-6cb78a0b560d None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:03.254959 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 291/581] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jul 25 12:02:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:02:03.254959 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78feb4e1-b107-4bad-bcc0-ede98883684c None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:03.265318 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:03.291234 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1902b36-6fa6-405b-9311-c95f72f7007c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:03.292002 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1902b36-6fa6-405b-9311-c95f72f7007c None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:03.292292 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1902b36-6fa6-405b-9311-c95f72f7007c None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:03.292570 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1902b36-6fa6-405b-9311-c95f72f7007c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:03.306063 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1902b36-6fa6-405b-9311-c95f72f7007c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:03.307519 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1902b36-6fa6-405b-9311-c95f72f7007c None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:03.325709 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:03.325922 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-d1902b36-6fa6-405b-9311-c95f72f7007c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:02:03.325922 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:02:03.325922 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:02:03.325922 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:02:03.325922 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:02:03.325922 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:02:03.325922 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:02:03.325922 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:02:03.325922 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:02:03.325922 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:02:03.325922 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:02:03.325922 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:02:03.325922 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 12:02:03.325922 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 12:02:03.325922 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 12:02:03.326781 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 12:02:03.326781 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:02:03.326781 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:02:03.326781 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:02:03.326781 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:02:03.326781 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:02:03.326781 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:02:03.326781 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:02:03.326781 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 25 12:02:03.326781 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:02:03.326781 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:02:03.326781 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:02:03.326781 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:02:03.326781 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:02:03.326781 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:02:03.326781 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:02:03.326781 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:02:03.328764 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 12:02:03.328764 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 12:02:03.328764 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:02:03.328764 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 25 12:02:03.328764 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 291/582] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jul 25 12:02:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:02:03.328764 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1902b36-6fa6-405b-9311-c95f72f7007c None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:03.360307 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78feb4e1-b107-4bad-bcc0-ede98883684c None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:03.396506 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733adabd-486f-4787-a081-e0149ae1b650 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:03.397530 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733adabd-486f-4787-a081-e0149ae1b650 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:03.397998 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733adabd-486f-4787-a081-e0149ae1b650 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:03.398457 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733adabd-486f-4787-a081-e0149ae1b650 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:03.409561 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733adabd-486f-4787-a081-e0149ae1b650 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:03.411191 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733adabd-486f-4787-a081-e0149ae1b650 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:03.429529 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-733adabd-486f-4787-a081-e0149ae1b650 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:02:03.429529 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:02:03.429529 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:02:03.429529 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:02:03.429529 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:02:03.429529 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:02:03.429529 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:02:03.429529 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:02:03.429529 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:02:03.429529 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:02:03.429529 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:02:03.429529 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:02:03.429529 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 12:02:03.429529 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 12:02:03.429529 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 12:02:03.430420 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 12:02:03.430420 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:02:03.430420 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:02:03.430420 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:02:03.430420 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:02:03.430420 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:02:03.430420 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:02:03.430420 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:02:03.430420 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jul 25 12:02:03.430420 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:02:03.430420 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:02:03.430420 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:02:03.430420 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:02:03.430420 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:02:03.430420 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:02:03.430420 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:02:03.430420 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:02:03.431483 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 12:02:03.431483 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 12:02:03.431483 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:02:03.431483 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jul 25 12:02:03.433017 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 292/583] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jul 25 12:02:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:02:03.433932 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733adabd-486f-4787-a081-e0149ae1b650 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:03.445013 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1902b36-6fa6-405b-9311-c95f72f7007c None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:03.479218 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02002175-2be2-4e33-9ab0-633abe94697a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:03.480043 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02002175-2be2-4e33-9ab0-633abe94697a None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:03.480326 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02002175-2be2-4e33-9ab0-633abe94697a None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:03.480602 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02002175-2be2-4e33-9ab0-633abe94697a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:03.491886 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02002175-2be2-4e33-9ab0-633abe94697a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:03.493322 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02002175-2be2-4e33-9ab0-633abe94697a None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:03.510759 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-02002175-2be2-4e33-9ab0-633abe94697a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:02:03.510759 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:02:03.510759 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:02:03.510759 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:02:03.510759 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:02:03.510759 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:02:03.510759 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:02:03.510759 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:02:03.510759 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:02:03.510759 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:02:03.510759 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:02:03.510759 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:02:03.510759 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 12:02:03.510759 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 12:02:03.510759 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 12:02:03.511906 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 12:02:03.511906 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:02:03.511906 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:02:03.511906 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:02:03.511906 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:02:03.511906 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:02:03.511906 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:02:03.511906 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:02:03.511906 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 25 12:02:03.511906 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:02:03.511906 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:02:03.511906 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:02:03.511906 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:02:03.511906 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:02:03.511906 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:02:03.511906 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:02:03.511906 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:02:03.512840 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 12:02:03.512840 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 12:02:03.512840 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:02:03.512840 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 25 12:02:03.512840 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 292/584] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jul 25 12:02:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:02:03.512840 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02002175-2be2-4e33-9ab0-633abe94697a None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:03.536724 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:03.546451 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733adabd-486f-4787-a081-e0149ae1b650 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:03.580180 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:03.593032 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03a3ed1-0a0f-4e3a-90ba-a3000bd851d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:03.593919 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03a3ed1-0a0f-4e3a-90ba-a3000bd851d3 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:03.594120 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03a3ed1-0a0f-4e3a-90ba-a3000bd851d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:03.594400 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03a3ed1-0a0f-4e3a-90ba-a3000bd851d3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:03.602081 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03a3ed1-0a0f-4e3a-90ba-a3000bd851d3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:03.603538 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03a3ed1-0a0f-4e3a-90ba-a3000bd851d3 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:03.618856 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-b03a3ed1-0a0f-4e3a-90ba-a3000bd851d3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 12:02:03.618856 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:02:03.618856 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:02:03.618856 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:02:03.618856 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:02:03.618856 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:02:03.618856 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:02:03.618856 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:02:03.618856 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:02:03.618856 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:02:03.618856 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:02:03.618856 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:02:03.618856 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 12:02:03.618856 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 12:02:03.618856 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 12:02:03.619879 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 12:02:03.619879 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:02:03.619879 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:02:03.619879 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:02:03.619879 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:02:03.619879 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:02:03.619879 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:02:03.619879 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:02:03.619879 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jul 25 12:02:03.619879 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:02:03.619879 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:02:03.619879 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:02:03.619879 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:02:03.619879 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:02:03.619879 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:02:03.619879 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:02:03.619879 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:02:03.620757 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 12:02:03.620757 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 12:02:03.620757 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 12:02:03.620757 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 12:02:03.620757 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 12:02:03.620757 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 12:02:03.620757 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 12:02:03.620757 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jul 25 12:02:03.620757 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 293/585] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Jul 25 12:02:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:02:03.621248 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03a3ed1-0a0f-4e3a-90ba-a3000bd851d3 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:03.627573 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02002175-2be2-4e33-9ab0-633abe94697a None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:03.665025 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2440c868-db89-4845-857a-a1bf2b82bd19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:03.665948 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2440c868-db89-4845-857a-a1bf2b82bd19 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:03.666242 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2440c868-db89-4845-857a-a1bf2b82bd19 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:03.666521 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2440c868-db89-4845-857a-a1bf2b82bd19 None admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:03.667253 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2440c868-db89-4845-857a-a1bf2b82bd19 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:02:03.667606 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2440c868-db89-4845-857a-a1bf2b82bd19 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:03.668989 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2440c868-db89-4845-857a-a1bf2b82bd19 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:03.682102 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 293/586] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jul 25 12:02:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:02:03.682833 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2440c868-db89-4845-857a-a1bf2b82bd19 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:03.737709 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03a3ed1-0a0f-4e3a-90ba-a3000bd851d3 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:03.764834 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:03.780180 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b8b8be-3afd-47e0-b13f-4bc02eed71fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:03.781011 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b8b8be-3afd-47e0-b13f-4bc02eed71fe None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:03.781309 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b8b8be-3afd-47e0-b13f-4bc02eed71fe None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:03.781913 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b8b8be-3afd-47e0-b13f-4bc02eed71fe None admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:03.782651 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b8b8be-3afd-47e0-b13f-4bc02eed71fe None admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:02:03.782974 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b8b8be-3afd-47e0-b13f-4bc02eed71fe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:03.785178 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b8b8be-3afd-47e0-b13f-4bc02eed71fe None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:03.791499 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2440c868-db89-4845-857a-a1bf2b82bd19 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:03.797850 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 294/587] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jul 25 12:02:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:02:03.798479 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b8b8be-3afd-47e0-b13f-4bc02eed71fe None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:03.831601 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a844d674-b585-43d7-9ad7-073759917605 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:03.832267 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a844d674-b585-43d7-9ad7-073759917605 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:03.832556 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a844d674-b585-43d7-9ad7-073759917605 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:03.832834 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a844d674-b585-43d7-9ad7-073759917605 None admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:03.833553 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a844d674-b585-43d7-9ad7-073759917605 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:02:03.833864 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a844d674-b585-43d7-9ad7-073759917605 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:03.835232 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a844d674-b585-43d7-9ad7-073759917605 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:03.842906 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:03.851699 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 294/588] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jul 25 12:02:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:02:03.852918 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a844d674-b585-43d7-9ad7-073759917605 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:03.902791 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b8b8be-3afd-47e0-b13f-4bc02eed71fe None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:03.936987 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80424ea4-fdfa-4163-a416-8fffadce8f29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:03.937651 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80424ea4-fdfa-4163-a416-8fffadce8f29 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:03.937939 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80424ea4-fdfa-4163-a416-8fffadce8f29 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:03.938219 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80424ea4-fdfa-4163-a416-8fffadce8f29 None admin] PATH_INFO: `/v3/projects` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:03.939008 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80424ea4-fdfa-4163-a416-8fffadce8f29 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:02:03.939322 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80424ea4-fdfa-4163-a416-8fffadce8f29 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:03.941517 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80424ea4-fdfa-4163-a416-8fffadce8f29 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:03.953300 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 295/589] 10.222.0.42 () {56 vars in 1148 bytes} [Mon Jul 25 12:02:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:02:03.954098 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80424ea4-fdfa-4163-a416-8fffadce8f29 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:03.973537 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a844d674-b585-43d7-9ad7-073759917605 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:04.013757 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b283505-7acc-47cd-9ea7-43c7469f21aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:04.014678 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b283505-7acc-47cd-9ea7-43c7469f21aa None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:04.015078 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b283505-7acc-47cd-9ea7-43c7469f21aa None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:04.015362 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b283505-7acc-47cd-9ea7-43c7469f21aa None admin] PATH_INFO: `/v3/projects/service` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:04.023015 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b283505-7acc-47cd-9ea7-43c7469f21aa None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:04.024679 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b283505-7acc-47cd-9ea7-43c7469f21aa None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:04.027690 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:04.043250 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-4b283505-7acc-47cd-9ea7-43c7469f21aa None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 12:02:04.043250 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:02:04.043250 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:02:04.043250 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:02:04.043250 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:02:04.043250 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:02:04.043250 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:02:04.043250 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:02:04.043250 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:02:04.043250 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:02:04.043250 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:02:04.043250 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:02:04.043250 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 12:02:04.043250 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 12:02:04.043250 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 12:02:04.044438 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 12:02:04.044438 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:02:04.044438 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:02:04.044438 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:02:04.044438 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:02:04.044438 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:02:04.044438 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:02:04.044438 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:02:04.044438 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 25 12:02:04.044438 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:02:04.044438 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:02:04.044438 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:02:04.044438 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:02:04.044438 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:02:04.044438 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:02:04.044438 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:02:04.044438 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:02:04.045347 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 12:02:04.045347 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 12:02:04.045347 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 12:02:04.045347 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 12:02:04.045347 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 12:02:04.045347 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 12:02:04.045347 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 12:02:04.045347 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 25 12:02:04.045347 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 295/590] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Jul 25 12:02:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:02:04.045347 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b283505-7acc-47cd-9ea7-43c7469f21aa None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:04.067443 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80424ea4-fdfa-4163-a416-8fffadce8f29 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:04.069603 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:04.108214 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f8cd09-bd52-4534-a766-07f6287d87da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:04.109015 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f8cd09-bd52-4534-a766-07f6287d87da None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:04.109302 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f8cd09-bd52-4534-a766-07f6287d87da None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:04.109603 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f8cd09-bd52-4534-a766-07f6287d87da None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:04.117773 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f8cd09-bd52-4534-a766-07f6287d87da None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:04.119228 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f8cd09-bd52-4534-a766-07f6287d87da None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:04.134126 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-f4f8cd09-bd52-4534-a766-07f6287d87da None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 25 12:02:04.134126 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:02:04.134126 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:02:04.134126 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:02:04.134126 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:02:04.134126 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:02:04.134126 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:02:04.134126 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:02:04.134126 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:02:04.134126 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:02:04.134126 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:02:04.134126 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:02:04.134126 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 12:02:04.134126 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 12:02:04.134126 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 12:02:04.134126 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 12:02:04.135083 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:02:04.135083 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:02:04.135083 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 12:02:04.135083 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 12:02:04.135083 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 12:02:04.135083 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 12:02:04.135083 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:02:04.135083 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:02:04.135083 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:02:04.135083 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:02:04.135083 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:02:04.135083 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jul 25 12:02:04.135083 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:02:04.135083 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:02:04.135083 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:02:04.135083 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:02:04.135083 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:02:04.135083 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:02:04.136087 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:02:04.136087 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:02:04.136087 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 12:02:04.136087 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 12:02:04.136087 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 12:02:04.136087 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 12:02:04.136087 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 12:02:04.136087 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 12:02:04.136087 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 25 12:02:04.136087 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jul 25 12:02:04.136087 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 296/591] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Jul 25 12:02:03 2022] GET /identity/v3/users/cinder => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:02:04.136087 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f8cd09-bd52-4534-a766-07f6287d87da None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:04.156178 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b283505-7acc-47cd-9ea7-43c7469f21aa None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:04.192950 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc80f545-9cdf-47bd-b155-5aefc985caa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:04.193853 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc80f545-9cdf-47bd-b155-5aefc985caa3 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:04.194542 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc80f545-9cdf-47bd-b155-5aefc985caa3 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:04.194542 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc80f545-9cdf-47bd-b155-5aefc985caa3 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:04.203622 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc80f545-9cdf-47bd-b155-5aefc985caa3 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:04.205982 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc80f545-9cdf-47bd-b155-5aefc985caa3 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:04.225407 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-bc80f545-9cdf-47bd-b155-5aefc985caa3 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 25 12:02:04.225407 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:02:04.225407 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:02:04.225407 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:02:04.225407 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:02:04.225407 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:02:04.225407 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:02:04.225407 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:02:04.225407 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:02:04.225407 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:02:04.225407 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:02:04.225407 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:02:04.225407 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 12:02:04.225407 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 12:02:04.225407 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 12:02:04.225407 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 12:02:04.226353 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:02:04.226353 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:02:04.226353 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 12:02:04.226353 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 12:02:04.226353 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 12:02:04.226353 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 12:02:04.226353 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:02:04.226353 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:02:04.226353 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:02:04.226353 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:02:04.226353 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:02:04.226353 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 25 12:02:04.226353 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:02:04.226353 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:02:04.226353 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:02:04.226353 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:02:04.226353 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:02:04.226353 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:02:04.227299 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:02:04.227299 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:02:04.227299 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 12:02:04.227299 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 12:02:04.227299 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 12:02:04.227299 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 12:02:04.227299 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 12:02:04.227299 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 12:02:04.227299 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 25 12:02:04.227299 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 25 12:02:04.227864 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 296/592] 10.222.0.42 () {56 vars in 1102 bytes} [Mon Jul 25 12:02:04 2022] GET /identity/v3/users/neutron => generated 85 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:02:04.228554 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc80f545-9cdf-47bd-b155-5aefc985caa3 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:04.249733 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f8cd09-bd52-4534-a766-07f6287d87da None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:04.286344 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c5c1bb-0278-4fcc-a3c2-8f4bf3aaa89f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:04.287246 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c5c1bb-0278-4fcc-a3c2-8f4bf3aaa89f None admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:04.287534 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c5c1bb-0278-4fcc-a3c2-8f4bf3aaa89f None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:04.287853 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c5c1bb-0278-4fcc-a3c2-8f4bf3aaa89f None admin] PATH_INFO: `/v3/projects/b9f62b8f37304b8a9a4bb54205347577/users/5ea16528b1374bae9cf54603f7907bfd/roles/79dec36aee5b4322bf652041c25ca6e4` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:04.295323 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:04.304355 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c5c1bb-0278-4fcc-a3c2-8f4bf3aaa89f None admin] RBAC: Authorizing `identity:create_grant(project_id=b9f62b8f37304b8a9a4bb54205347577, user_id=5ea16528b1374bae9cf54603f7907bfd, role_id=79dec36aee5b4322bf652041c25ca6e4)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:04.305792 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c5c1bb-0278-4fcc-a3c2-8f4bf3aaa89f None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:04.319024 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 297/593] 10.222.0.42 () {58 vars in 1439 bytes} [Mon Jul 25 12:02:04 2022] PUT /identity/v3/projects/b9f62b8f37304b8a9a4bb54205347577/users/5ea16528b1374bae9cf54603f7907bfd/roles/79dec36aee5b4322bf652041c25ca6e4 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:02:04.319780 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c5c1bb-0278-4fcc-a3c2-8f4bf3aaa89f None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:04.345550 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:04.362891 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc80f545-9cdf-47bd-b155-5aefc985caa3 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:04.412444 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e222d2-91ef-4f04-af6e-5fe8e89cfa07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:04.413686 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e222d2-91ef-4f04-af6e-5fe8e89cfa07 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:04.414847 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e222d2-91ef-4f04-af6e-5fe8e89cfa07 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:04.415366 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e222d2-91ef-4f04-af6e-5fe8e89cfa07 None admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:04.417001 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e222d2-91ef-4f04-af6e-5fe8e89cfa07 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:02:04.417787 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e222d2-91ef-4f04-af6e-5fe8e89cfa07 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:04.420679 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e222d2-91ef-4f04-af6e-5fe8e89cfa07 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:04.438362 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 297/594] 10.222.0.42 () {56 vars in 1148 bytes} [Mon Jul 25 12:02:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:02:04.439676 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e222d2-91ef-4f04-af6e-5fe8e89cfa07 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:04.457015 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c5c1bb-0278-4fcc-a3c2-8f4bf3aaa89f None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:04.498694 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535a29ab-ebca-4a64-834a-016abbc425bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:04.499437 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535a29ab-ebca-4a64-834a-016abbc425bd None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:04.499757 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535a29ab-ebca-4a64-834a-016abbc425bd None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:04.500039 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535a29ab-ebca-4a64-834a-016abbc425bd None admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:04.500958 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535a29ab-ebca-4a64-834a-016abbc425bd None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:02:04.501276 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535a29ab-ebca-4a64-834a-016abbc425bd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:04.502934 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535a29ab-ebca-4a64-834a-016abbc425bd None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:04.525033 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 298/595] 10.222.0.42 () {56 vars in 1137 bytes} [Mon Jul 25 12:02:04 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:02:04.525696 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535a29ab-ebca-4a64-834a-016abbc425bd None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:04.568567 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e222d2-91ef-4f04-af6e-5fe8e89cfa07 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:04.588811 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:04.600627 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:04.605424 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b892d8fb-3e4e-4e55-a616-123b25849e18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:04.606109 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b892d8fb-3e4e-4e55-a616-123b25849e18 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:04.606397 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b892d8fb-3e4e-4e55-a616-123b25849e18 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:04.606678 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b892d8fb-3e4e-4e55-a616-123b25849e18 None admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:04.607531 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b892d8fb-3e4e-4e55-a616-123b25849e18 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:02:04.607874 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b892d8fb-3e4e-4e55-a616-123b25849e18 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:04.609309 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b892d8fb-3e4e-4e55-a616-123b25849e18 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:04.627037 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 298/596] 10.222.0.42 () {56 vars in 1139 bytes} [Mon Jul 25 12:02:04 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:02:04.628593 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d863426-b08e-4609-b989-bc236f2a9426 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:04.629056 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d863426-b08e-4609-b989-bc236f2a9426 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:04.629507 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d863426-b08e-4609-b989-bc236f2a9426 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:04.634960 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 299/597] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 25 12:02:04 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:02:04.635754 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d863426-b08e-4609-b989-bc236f2a9426 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:04.643338 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535a29ab-ebca-4a64-834a-016abbc425bd None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:04.678855 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b52bee8-fe2e-4da3-86d3-e3e36c2d2ba1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:04.679803 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b52bee8-fe2e-4da3-86d3-e3e36c2d2ba1 None admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:04.680095 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b52bee8-fe2e-4da3-86d3-e3e36c2d2ba1 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:04.680380 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b52bee8-fe2e-4da3-86d3-e3e36c2d2ba1 None admin] PATH_INFO: `/v3/projects/b9f62b8f37304b8a9a4bb54205347577/users/f67142a1193d47c7bd33d7040268233c/roles/79dec36aee5b4322bf652041c25ca6e4` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:04.696359 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b52bee8-fe2e-4da3-86d3-e3e36c2d2ba1 None admin] RBAC: Authorizing `identity:create_grant(project_id=b9f62b8f37304b8a9a4bb54205347577, user_id=f67142a1193d47c7bd33d7040268233c, role_id=79dec36aee5b4322bf652041c25ca6e4)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:04.697834 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b52bee8-fe2e-4da3-86d3-e3e36c2d2ba1 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:04.712419 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 299/598] 10.222.0.42 () {58 vars in 1439 bytes} [Mon Jul 25 12:02:04 2022] PUT /identity/v3/projects/b9f62b8f37304b8a9a4bb54205347577/users/f67142a1193d47c7bd33d7040268233c/roles/79dec36aee5b4322bf652041c25ca6e4 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:02:04.720614 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b52bee8-fe2e-4da3-86d3-e3e36c2d2ba1 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:04.761451 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d863426-b08e-4609-b989-bc236f2a9426 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:04.813112 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ca19f8-3eb2-4b81-ab5d-9934d122fea2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:04.813426 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ca19f8-3eb2-4b81-ab5d-9934d122fea2 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:04.813839 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ca19f8-3eb2-4b81-ab5d-9934d122fea2 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:04.814275 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ca19f8-3eb2-4b81-ab5d-9934d122fea2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:04.827428 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ca19f8-3eb2-4b81-ab5d-9934d122fea2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:04.829478 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ca19f8-3eb2-4b81-ab5d-9934d122fea2 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:04.849678 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-b5ca19f8-3eb2-4b81-ab5d-9934d122fea2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:02:04.849678 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:02:04.849678 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:02:04.849678 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:02:04.849678 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:02:04.849678 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:02:04.849678 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:02:04.849678 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:02:04.849678 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:02:04.849678 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:02:04.849678 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:02:04.849678 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:02:04.849678 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 12:02:04.849678 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 12:02:04.849678 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 12:02:04.850773 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 12:02:04.850773 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:02:04.850773 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:02:04.850773 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:02:04.850773 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:02:04.850773 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:02:04.850773 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:02:04.850773 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:02:04.850773 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 25 12:02:04.850773 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:02:04.850773 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:02:04.850773 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:02:04.850773 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:02:04.850773 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:02:04.850773 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:02:04.850773 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:02:04.850773 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:02:04.851873 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 12:02:04.851873 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 12:02:04.851873 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:02:04.851873 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 25 12:02:04.857565 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 300/599] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jul 25 12:02:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:02:04.857565 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1f34e8-394a-4ceb-8739-48e534a91efb None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:04.857565 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1f34e8-394a-4ceb-8739-48e534a91efb None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:04.857565 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1f34e8-394a-4ceb-8739-48e534a91efb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:04.860238 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b52bee8-fe2e-4da3-86d3-e3e36c2d2ba1 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:04.877793 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-1a1f34e8-394a-4ceb-8739-48e534a91efb None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:02:04.879260 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-1a1f34e8-394a-4ceb-8739-48e534a91efb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:02:04.887397 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:04.907059 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:04.913653 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bc6946-dbd6-4d23-960e-3560f20f91b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:04.914688 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bc6946-dbd6-4d23-960e-3560f20f91b9 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:04.914982 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bc6946-dbd6-4d23-960e-3560f20f91b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:04.915431 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bc6946-dbd6-4d23-960e-3560f20f91b9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:04.927579 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13bc6946-dbd6-4d23-960e-3560f20f91b9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:04.929069 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13bc6946-dbd6-4d23-960e-3560f20f91b9 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:04.947596 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-13bc6946-dbd6-4d23-960e-3560f20f91b9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:02:04.947596 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:02:04.947596 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:02:04.947596 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:02:04.947596 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:02:04.947596 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:02:04.947596 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:02:04.947596 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:02:04.947596 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:02:04.947596 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:02:04.947596 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:02:04.947596 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:02:04.947596 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 12:02:04.947596 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 12:02:04.947596 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 12:02:04.948510 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 12:02:04.948510 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:02:04.948510 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:02:04.948510 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:02:04.948510 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:02:04.948510 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:02:04.948510 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:02:04.948510 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:02:04.948510 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jul 25 12:02:04.948510 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:02:04.948510 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:02:04.948510 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:02:04.948510 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:02:04.948510 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:02:04.948510 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:02:04.948510 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:02:04.948510 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:02:04.949712 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 12:02:04.949712 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 12:02:04.949712 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:02:04.949712 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jul 25 12:02:04.949712 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 300/600] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jul 25 12:02:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:02:04.949712 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bc6946-dbd6-4d23-960e-3560f20f91b9 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:04.967576 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 301/601] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 25 12:02:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 12:02:04.968748 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1f34e8-394a-4ceb-8739-48e534a91efb None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:05.080294 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bc6946-dbd6-4d23-960e-3560f20f91b9 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:05.108048 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1f34e8-394a-4ceb-8739-48e534a91efb None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:05.117687 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f99120-48d4-4768-b907-909107a8e08d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:05.118440 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f99120-48d4-4768-b907-909107a8e08d None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:05.118798 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f99120-48d4-4768-b907-909107a8e08d None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:05.119149 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f99120-48d4-4768-b907-909107a8e08d None admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:05.120042 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f99120-48d4-4768-b907-909107a8e08d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:02:05.120431 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f99120-48d4-4768-b907-909107a8e08d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:05.121875 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f99120-48d4-4768-b907-909107a8e08d None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:05.134239 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 301/602] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jul 25 12:02:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:02:05.135147 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f99120-48d4-4768-b907-909107a8e08d None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:05.157925 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba442ae7-b31b-41bd-9258-581e7958742d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:05.159120 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba442ae7-b31b-41bd-9258-581e7958742d None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:05.159932 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba442ae7-b31b-41bd-9258-581e7958742d None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:05.160220 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba442ae7-b31b-41bd-9258-581e7958742d None admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:05.161213 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba442ae7-b31b-41bd-9258-581e7958742d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:02:05.161527 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba442ae7-b31b-41bd-9258-581e7958742d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:05.163558 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba442ae7-b31b-41bd-9258-581e7958742d None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:05.170005 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:05.183633 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 302/603] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jul 25 12:02:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:02:05.184512 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba442ae7-b31b-41bd-9258-581e7958742d None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:05.211702 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:05.261410 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f99120-48d4-4768-b907-909107a8e08d None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:05.299650 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165a336e-52f5-4046-9a00-1cef760e16b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:05.300495 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165a336e-52f5-4046-9a00-1cef760e16b9 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:05.300791 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165a336e-52f5-4046-9a00-1cef760e16b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:05.301074 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165a336e-52f5-4046-9a00-1cef760e16b9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:05.314105 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165a336e-52f5-4046-9a00-1cef760e16b9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 12:02:05.319099 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba442ae7-b31b-41bd-9258-581e7958742d None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:05.324052 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-165a336e-52f5-4046-9a00-1cef760e16b9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 25 12:02:05.324052 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:02:05.324052 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:02:05.324052 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:02:05.324052 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:02:05.324052 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:02:05.324052 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:02:05.324052 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:02:05.324052 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:02:05.324052 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:02:05.324052 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:02:05.324052 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:02:05.324052 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 12:02:05.324052 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 12:02:05.324052 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 12:02:05.324052 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 12:02:05.326215 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 12:02:05.326215 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 12:02:05.326215 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 12:02:05.326215 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 12:02:05.326215 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:02:05.326215 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:02:05.326215 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:02:05.326215 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:02:05.326215 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:02:05.326215 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:02:05.326215 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:02:05.326215 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jul 25 12:02:05.326215 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:02:05.326215 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:02:05.326215 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:02:05.326215 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:02:05.327640 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:02:05.327640 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:02:05.327640 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:02:05.327640 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:02:05.327640 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 12:02:05.327640 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 12:02:05.327640 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 12:02:05.327640 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 12:02:05.327640 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 12:02:05.327640 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 12:02:05.327640 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 25 12:02:05.327640 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jul 25 12:02:05.327640 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 302/604] 10.222.0.42 () {56 vars in 1102 bytes} [Mon Jul 25 12:02:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:02:05.327640 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165a336e-52f5-4046-9a00-1cef760e16b9 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:05.365081 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c0a277-2897-4f4a-9875-968173dd1b1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:05.366489 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c0a277-2897-4f4a-9875-968173dd1b1c None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:05.366783 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c0a277-2897-4f4a-9875-968173dd1b1c None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:05.367065 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c0a277-2897-4f4a-9875-968173dd1b1c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:05.376302 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c0a277-2897-4f4a-9875-968173dd1b1c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:05.377789 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c0a277-2897-4f4a-9875-968173dd1b1c None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:05.395119 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-d9c0a277-2897-4f4a-9875-968173dd1b1c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 12:02:05.395119 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:02:05.395119 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:02:05.395119 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:02:05.395119 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:02:05.395119 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:02:05.395119 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:02:05.395119 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:02:05.395119 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:02:05.395119 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:02:05.395119 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:02:05.395119 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:02:05.395119 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 12:02:05.395119 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 12:02:05.395119 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 12:02:05.396501 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 12:02:05.396501 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:02:05.396501 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:02:05.396501 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:02:05.396501 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:02:05.396501 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:02:05.396501 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:02:05.396501 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:02:05.396501 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 25 12:02:05.396501 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:02:05.396501 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:02:05.396501 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:02:05.396501 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:02:05.396501 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:02:05.396501 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:02:05.396501 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:02:05.396501 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:02:05.398505 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 12:02:05.398505 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 12:02:05.398505 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 12:02:05.398505 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 12:02:05.398505 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 12:02:05.398505 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 12:02:05.398505 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 12:02:05.398505 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 25 12:02:05.398505 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 303/605] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Jul 25 12:02:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:02:05.399799 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c0a277-2897-4f4a-9875-968173dd1b1c None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:05.446777 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:05.458101 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165a336e-52f5-4046-9a00-1cef760e16b9 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:05.497073 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f908c0b-ae26-4de0-9a99-e5cc35f993bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:05.497889 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f908c0b-ae26-4de0-9a99-e5cc35f993bd None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:05.498179 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f908c0b-ae26-4de0-9a99-e5cc35f993bd None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:05.498560 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f908c0b-ae26-4de0-9a99-e5cc35f993bd None admin] PATH_INFO: `/v3/projects/service` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:05.505465 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:05.507119 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f908c0b-ae26-4de0-9a99-e5cc35f993bd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:05.509035 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f908c0b-ae26-4de0-9a99-e5cc35f993bd None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:05.528272 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c0a277-2897-4f4a-9875-968173dd1b1c None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:05.529199 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-6f908c0b-ae26-4de0-9a99-e5cc35f993bd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 12:02:05.529199 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:02:05.529199 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:02:05.529199 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:02:05.529199 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:02:05.529199 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:02:05.529199 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:02:05.529199 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:02:05.529199 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:02:05.529199 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:02:05.529199 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:02:05.529199 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:02:05.529199 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 12:02:05.529199 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 12:02:05.529199 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 12:02:05.531145 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 12:02:05.531145 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:02:05.531145 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:02:05.531145 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:02:05.531145 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:02:05.531145 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:02:05.531145 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:02:05.531145 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:02:05.531145 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jul 25 12:02:05.531145 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:02:05.531145 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:02:05.531145 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:02:05.531145 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:02:05.531145 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:02:05.531145 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:02:05.531145 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:02:05.531145 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:02:05.534128 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 12:02:05.534128 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 12:02:05.534128 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 12:02:05.534128 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 12:02:05.534128 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 12:02:05.534128 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 12:02:05.534128 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 12:02:05.534128 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jul 25 12:02:05.534128 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 303/606] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Jul 25 12:02:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:02:05.534128 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f908c0b-ae26-4de0-9a99-e5cc35f993bd None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:05.577210 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd24ae78-69cc-4d2c-bdf3-b129915ae40e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:05.577890 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd24ae78-69cc-4d2c-bdf3-b129915ae40e None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:05.578175 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd24ae78-69cc-4d2c-bdf3-b129915ae40e None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:05.578456 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd24ae78-69cc-4d2c-bdf3-b129915ae40e None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:05.579456 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd24ae78-69cc-4d2c-bdf3-b129915ae40e None admin] RBAC: Adding query filter params (name=service) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:02:05.581156 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd24ae78-69cc-4d2c-bdf3-b129915ae40e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:05.582649 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd24ae78-69cc-4d2c-bdf3-b129915ae40e None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:05.592892 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 304/607] 10.222.0.42 () {56 vars in 1103 bytes} [Mon Jul 25 12:02:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:02:05.596529 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd24ae78-69cc-4d2c-bdf3-b129915ae40e None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:05.674105 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f908c0b-ae26-4de0-9a99-e5cc35f993bd None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:05.717623 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0442d09f-60fa-4353-9354-0f8f15c22102 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:05.718672 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0442d09f-60fa-4353-9354-0f8f15c22102 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:05.719097 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0442d09f-60fa-4353-9354-0f8f15c22102 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:05.719510 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0442d09f-60fa-4353-9354-0f8f15c22102 None admin] PATH_INFO: `/v3/projects` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:05.720481 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0442d09f-60fa-4353-9354-0f8f15c22102 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:02:05.720830 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0442d09f-60fa-4353-9354-0f8f15c22102 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:05.722931 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0442d09f-60fa-4353-9354-0f8f15c22102 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:05.727481 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd24ae78-69cc-4d2c-bdf3-b129915ae40e None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:05.737069 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 304/608] 10.222.0.42 () {56 vars in 1148 bytes} [Mon Jul 25 12:02:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:02:05.737886 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0442d09f-60fa-4353-9354-0f8f15c22102 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:05.761254 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:05.766911 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052fc91b-8a82-465a-bb01-2cdd16fdeb16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:05.768175 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052fc91b-8a82-465a-bb01-2cdd16fdeb16 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:05.768476 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052fc91b-8a82-465a-bb01-2cdd16fdeb16 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:05.768759 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052fc91b-8a82-465a-bb01-2cdd16fdeb16 None admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:05.769553 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052fc91b-8a82-465a-bb01-2cdd16fdeb16 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:02:05.769867 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052fc91b-8a82-465a-bb01-2cdd16fdeb16 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:05.771316 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052fc91b-8a82-465a-bb01-2cdd16fdeb16 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:05.787735 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 305/609] 10.222.0.42 () {56 vars in 1148 bytes} [Mon Jul 25 12:02:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:02:05.787735 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052fc91b-8a82-465a-bb01-2cdd16fdeb16 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:05.788024 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:05.870801 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0442d09f-60fa-4353-9354-0f8f15c22102 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:05.888707 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052fc91b-8a82-465a-bb01-2cdd16fdeb16 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:05.909574 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08b90af-85ae-4e18-92be-2c78e3903244 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:05.910400 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08b90af-85ae-4e18-92be-2c78e3903244 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:05.910688 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08b90af-85ae-4e18-92be-2c78e3903244 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:05.910967 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08b90af-85ae-4e18-92be-2c78e3903244 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:05.924389 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08b90af-85ae-4e18-92be-2c78e3903244 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:05.925984 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08b90af-85ae-4e18-92be-2c78e3903244 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:05.927179 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f8aeac-a5e1-4f8c-acdc-7738c4ee2f35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:05.928187 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f8aeac-a5e1-4f8c-acdc-7738c4ee2f35 None admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:05.928537 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f8aeac-a5e1-4f8c-acdc-7738c4ee2f35 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:05.928887 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f8aeac-a5e1-4f8c-acdc-7738c4ee2f35 None admin] PATH_INFO: `/v3/projects/b9f62b8f37304b8a9a4bb54205347577/users/6f89ce6f0cf04373a414d927fa0f8a7f/roles/79dec36aee5b4322bf652041c25ca6e4` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:05.944763 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f8aeac-a5e1-4f8c-acdc-7738c4ee2f35 None admin] RBAC: Authorizing `identity:create_grant(project_id=b9f62b8f37304b8a9a4bb54205347577, user_id=6f89ce6f0cf04373a414d927fa0f8a7f, role_id=79dec36aee5b4322bf652041c25ca6e4)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:05.945715 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-f08b90af-85ae-4e18-92be-2c78e3903244 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:02:05.945715 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:02:05.945715 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:02:05.945715 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:02:05.945715 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:02:05.945715 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:02:05.945715 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:02:05.945715 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:02:05.945715 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:02:05.945715 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:02:05.945715 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:02:05.945715 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:02:05.945715 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 12:02:05.945715 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 12:02:05.945715 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 12:02:05.946598 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 12:02:05.946598 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:02:05.946598 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:02:05.946598 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:02:05.946598 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:02:05.946598 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:02:05.946598 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:02:05.946598 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:02:05.946598 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jul 25 12:02:05.946598 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:02:05.946598 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:02:05.946598 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:02:05.946598 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:02:05.946598 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:02:05.946598 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:02:05.946598 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:02:05.946598 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:02:05.947479 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 12:02:05.947479 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 12:02:05.947479 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:02:05.947479 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jul 25 12:02:05.947754 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f8aeac-a5e1-4f8c-acdc-7738c4ee2f35 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:05.951733 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 305/610] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jul 25 12:02:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:02:05.952767 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08b90af-85ae-4e18-92be-2c78e3903244 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:05.965015 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 306/611] 10.222.0.42 () {58 vars in 1439 bytes} [Mon Jul 25 12:02:05 2022] PUT /identity/v3/projects/b9f62b8f37304b8a9a4bb54205347577/users/6f89ce6f0cf04373a414d927fa0f8a7f/roles/79dec36aee5b4322bf652041c25ca6e4 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:02:05.966053 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f8aeac-a5e1-4f8c-acdc-7738c4ee2f35 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:06.028513 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:06.086102 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f8aeac-a5e1-4f8c-acdc-7738c4ee2f35 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:06.086517 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:06.132510 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb3db24-d9a8-41d7-b780-5022f0c316cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:06.133209 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb3db24-d9a8-41d7-b780-5022f0c316cc None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:06.133496 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb3db24-d9a8-41d7-b780-5022f0c316cc None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:06.133775 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb3db24-d9a8-41d7-b780-5022f0c316cc None admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:06.134517 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb3db24-d9a8-41d7-b780-5022f0c316cc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:02:06.134826 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb3db24-d9a8-41d7-b780-5022f0c316cc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:06.136261 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb3db24-d9a8-41d7-b780-5022f0c316cc None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:06.162244 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 307/612] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jul 25 12:02:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:02:06.169053 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb3db24-d9a8-41d7-b780-5022f0c316cc None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:06.172034 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08b90af-85ae-4e18-92be-2c78e3903244 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:06.219131 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245b7d0b-dc54-4b3c-8d87-726fb86cbaaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:06.220121 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245b7d0b-dc54-4b3c-8d87-726fb86cbaaa None admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:06.220466 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245b7d0b-dc54-4b3c-8d87-726fb86cbaaa None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:06.221369 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245b7d0b-dc54-4b3c-8d87-726fb86cbaaa None admin] PATH_INFO: `/v3/projects/b9f62b8f37304b8a9a4bb54205347577/users/51733781201243e8abf9eca39b207750/roles/79dec36aee5b4322bf652041c25ca6e4` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:06.237836 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245b7d0b-dc54-4b3c-8d87-726fb86cbaaa None admin] RBAC: Authorizing `identity:create_grant(project_id=b9f62b8f37304b8a9a4bb54205347577, user_id=51733781201243e8abf9eca39b207750, role_id=79dec36aee5b4322bf652041c25ca6e4)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:06.239344 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245b7d0b-dc54-4b3c-8d87-726fb86cbaaa None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:06.262392 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 306/613] 10.222.0.42 () {58 vars in 1439 bytes} [Mon Jul 25 12:02:05 2022] PUT /identity/v3/projects/b9f62b8f37304b8a9a4bb54205347577/users/51733781201243e8abf9eca39b207750/roles/79dec36aee5b4322bf652041c25ca6e4 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:02:06.279825 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb3db24-d9a8-41d7-b780-5022f0c316cc None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:06.295399 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:06.320371 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe53bf9-ed51-4401-a364-70e4e03c448a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:06.321192 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe53bf9-ed51-4401-a364-70e4e03c448a None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:06.321482 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe53bf9-ed51-4401-a364-70e4e03c448a None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:06.321760 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe53bf9-ed51-4401-a364-70e4e03c448a None admin] PATH_INFO: `/v3/users/swift` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:06.330964 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe53bf9-ed51-4401-a364-70e4e03c448a None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:06.332463 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe53bf9-ed51-4401-a364-70e4e03c448a None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:06.348361 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-2fe53bf9-ed51-4401-a364-70e4e03c448a None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 25 12:02:06.348361 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:02:06.348361 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:02:06.348361 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:02:06.348361 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:02:06.348361 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:02:06.348361 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:02:06.348361 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:02:06.348361 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:02:06.348361 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:02:06.348361 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:02:06.348361 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:02:06.348361 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 12:02:06.348361 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 12:02:06.348361 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 12:02:06.348361 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 12:02:06.349374 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:02:06.349374 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:02:06.349374 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 12:02:06.349374 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 12:02:06.349374 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 12:02:06.349374 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 12:02:06.349374 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:02:06.349374 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:02:06.349374 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:02:06.349374 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:02:06.349374 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:02:06.349374 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 25 12:02:06.349374 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:02:06.349374 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:02:06.349374 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:02:06.349374 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:02:06.349374 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:02:06.349374 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:02:06.350300 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:02:06.350300 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:02:06.350300 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 12:02:06.350300 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 12:02:06.350300 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 12:02:06.350300 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 12:02:06.350300 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 12:02:06.350300 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 12:02:06.350300 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 25 12:02:06.350300 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 25 12:02:06.350300 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 308/614] 10.222.0.42 () {56 vars in 1096 bytes} [Mon Jul 25 12:02:06 2022] GET /identity/v3/users/swift => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 12:02:06.355285 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245b7d0b-dc54-4b3c-8d87-726fb86cbaaa None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:06.491236 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245b7d0b-dc54-4b3c-8d87-726fb86cbaaa None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:06.507140 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:06.529508 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e78fc2-5937-4311-950b-47b0ce374bac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:06.530209 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e78fc2-5937-4311-950b-47b0ce374bac None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:06.530499 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e78fc2-5937-4311-950b-47b0ce374bac None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:06.530785 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e78fc2-5937-4311-950b-47b0ce374bac None admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:06.531668 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e78fc2-5937-4311-950b-47b0ce374bac None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:02:06.531995 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e78fc2-5937-4311-950b-47b0ce374bac None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:06.533989 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e78fc2-5937-4311-950b-47b0ce374bac None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:06.556373 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 307/615] 10.222.0.42 () {56 vars in 1135 bytes} [Mon Jul 25 12:02:06 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:02:06.561916 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe53bf9-ed51-4401-a364-70e4e03c448a None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:06.633803 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1a2598-8288-4c3b-a403-79c99aa38d91 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:06.634110 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1a2598-8288-4c3b-a403-79c99aa38d91 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:06.634361 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1a2598-8288-4c3b-a403-79c99aa38d91 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:06.636121 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 308/616] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 25 12:02:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 12:02:06.641978 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfc613d-49cd-48b1-9760-1d97d1798665 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:06.642272 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfc613d-49cd-48b1-9760-1d97d1798665 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:06.642606 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfc613d-49cd-48b1-9760-1d97d1798665 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:06.662765 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-0cfc613d-49cd-48b1-9760-1d97d1798665 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:02:06.663996 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-0cfc613d-49cd-48b1-9760-1d97d1798665 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:02:06.685487 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe53bf9-ed51-4401-a364-70e4e03c448a None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:06.726014 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5df11a-509c-446e-bae9-f6ed10180550 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:06.727171 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5df11a-509c-446e-bae9-f6ed10180550 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:06.727711 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5df11a-509c-446e-bae9-f6ed10180550 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:06.728261 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5df11a-509c-446e-bae9-f6ed10180550 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:06.741404 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5df11a-509c-446e-bae9-f6ed10180550 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:06.741563 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 309/617] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 25 12:02:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 12:02:06.743252 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5df11a-509c-446e-bae9-f6ed10180550 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:06.761593 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-ff5df11a-509c-446e-bae9-f6ed10180550 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:02:06.761593 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:02:06.761593 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:02:06.761593 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:02:06.761593 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:02:06.761593 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:02:06.761593 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:02:06.761593 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:02:06.761593 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:02:06.761593 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:02:06.761593 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:02:06.761593 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:02:06.761593 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 12:02:06.761593 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 12:02:06.761593 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 12:02:06.762508 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 12:02:06.762508 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:02:06.762508 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:02:06.762508 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:02:06.762508 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:02:06.762508 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:02:06.762508 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:02:06.762508 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:02:06.762508 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 25 12:02:06.762508 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:02:06.762508 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:02:06.762508 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:02:06.762508 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:02:06.762508 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:02:06.762508 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:02:06.762508 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:02:06.762508 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:02:06.763416 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 12:02:06.763416 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 12:02:06.763416 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:02:06.763416 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 25 12:02:06.765736 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 309/618] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jul 25 12:02:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:02:06.771555 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfc613d-49cd-48b1-9760-1d97d1798665 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:06.778008 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5df11a-509c-446e-bae9-f6ed10180550 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:06.821669 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:06.890770 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfc613d-49cd-48b1-9760-1d97d1798665 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:06.893476 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:06.928936 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5df11a-509c-446e-bae9-f6ed10180550 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:06.932869 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59436433-ccb5-4fc8-a429-a0716f884c16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:06.933880 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59436433-ccb5-4fc8-a429-a0716f884c16 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:06.934390 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59436433-ccb5-4fc8-a429-a0716f884c16 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:06.934874 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59436433-ccb5-4fc8-a429-a0716f884c16 None admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:06.935861 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59436433-ccb5-4fc8-a429-a0716f884c16 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:02:06.936500 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59436433-ccb5-4fc8-a429-a0716f884c16 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:06.938105 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59436433-ccb5-4fc8-a429-a0716f884c16 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:06.951937 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 310/619] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jul 25 12:02:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:02:06.957654 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59436433-ccb5-4fc8-a429-a0716f884c16 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:06.978451 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c77e13b-2761-4076-b3b7-31df63744435 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:06.979383 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c77e13b-2761-4076-b3b7-31df63744435 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:06.979713 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c77e13b-2761-4076-b3b7-31df63744435 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:06.979998 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c77e13b-2761-4076-b3b7-31df63744435 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:07.006610 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c77e13b-2761-4076-b3b7-31df63744435 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 12:02:07.014586 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-3c77e13b-2761-4076-b3b7-31df63744435 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 25 12:02:07.014586 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:02:07.014586 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:02:07.014586 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:02:07.014586 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:02:07.014586 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:02:07.014586 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:02:07.014586 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:02:07.014586 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:02:07.014586 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:02:07.014586 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:02:07.014586 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:02:07.014586 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 12:02:07.014586 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 12:02:07.014586 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 12:02:07.014586 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 12:02:07.015704 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 12:02:07.015704 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 12:02:07.015704 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 12:02:07.015704 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 12:02:07.015704 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:02:07.015704 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:02:07.015704 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:02:07.015704 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:02:07.015704 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:02:07.015704 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:02:07.015704 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:02:07.015704 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 25 12:02:07.015704 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:02:07.015704 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:02:07.015704 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:02:07.015704 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:02:07.016538 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:02:07.016538 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:02:07.016538 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:02:07.016538 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:02:07.016538 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 12:02:07.016538 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 12:02:07.016538 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 12:02:07.016538 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 12:02:07.016538 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 12:02:07.016538 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 12:02:07.016538 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 25 12:02:07.016538 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 25 12:02:07.016538 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 310/620] 10.222.0.42 () {56 vars in 1102 bytes} [Mon Jul 25 12:02:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:02:07.018237 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9637bfcc-73fd-4fce-8325-92ed7a9bf704 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:07.018237 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9637bfcc-73fd-4fce-8325-92ed7a9bf704 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:07.018474 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9637bfcc-73fd-4fce-8325-92ed7a9bf704 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:07.024531 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 311/621] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 25 12:02:07 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:02:07.025304 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9637bfcc-73fd-4fce-8325-92ed7a9bf704 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:07.075643 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59436433-ccb5-4fc8-a429-a0716f884c16 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:07.112040 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53854481-7650-494b-9ec9-642f744d459d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:07.112879 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53854481-7650-494b-9ec9-642f744d459d None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:07.113217 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53854481-7650-494b-9ec9-642f744d459d None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:07.113583 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53854481-7650-494b-9ec9-642f744d459d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:07.120829 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53854481-7650-494b-9ec9-642f744d459d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:07.122413 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53854481-7650-494b-9ec9-642f744d459d None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:07.131890 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:07.138749 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-53854481-7650-494b-9ec9-642f744d459d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 12:02:07.138749 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:02:07.138749 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:02:07.138749 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:02:07.138749 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:02:07.138749 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:02:07.138749 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:02:07.138749 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:02:07.138749 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:02:07.138749 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:02:07.138749 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:02:07.138749 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:02:07.138749 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 12:02:07.138749 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 12:02:07.138749 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 12:02:07.139717 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 12:02:07.139717 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:02:07.139717 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:02:07.139717 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:02:07.139717 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:02:07.139717 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:02:07.139717 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:02:07.139717 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:02:07.139717 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jul 25 12:02:07.139717 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:02:07.139717 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:02:07.139717 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:02:07.139717 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:02:07.139717 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:02:07.139717 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:02:07.139717 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:02:07.139717 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:02:07.140600 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 12:02:07.140600 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 12:02:07.140600 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 12:02:07.140600 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 12:02:07.140600 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 12:02:07.140600 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 12:02:07.140600 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 12:02:07.140600 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jul 25 12:02:07.145085 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 311/622] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Jul 25 12:02:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:02:07.147471 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fb9a17-23d3-415a-9479-8ce122baa3da None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:07.147635 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fb9a17-23d3-415a-9479-8ce122baa3da None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:07.147848 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fb9a17-23d3-415a-9479-8ce122baa3da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:07.154537 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9637bfcc-73fd-4fce-8325-92ed7a9bf704 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:07.172013 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-35fb9a17-23d3-415a-9479-8ce122baa3da None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:02:07.173266 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-35fb9a17-23d3-415a-9479-8ce122baa3da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:02:07.194680 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:07.196740 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e89d211-0da2-44c7-81a2-8edf8673d1a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:07.197646 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e89d211-0da2-44c7-81a2-8edf8673d1a1 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:07.198298 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e89d211-0da2-44c7-81a2-8edf8673d1a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:07.199058 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e89d211-0da2-44c7-81a2-8edf8673d1a1 None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:07.200695 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e89d211-0da2-44c7-81a2-8edf8673d1a1 None admin] RBAC: Adding query filter params (name=service) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:02:07.202058 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e89d211-0da2-44c7-81a2-8edf8673d1a1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:07.204413 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e89d211-0da2-44c7-81a2-8edf8673d1a1 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:07.213976 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 312/623] 10.222.0.42 () {56 vars in 1103 bytes} [Mon Jul 25 12:02:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:02:07.215238 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e89d211-0da2-44c7-81a2-8edf8673d1a1 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:07.259692 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 312/624] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 25 12:02:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 12:02:07.260859 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fb9a17-23d3-415a-9479-8ce122baa3da None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:07.342324 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e89d211-0da2-44c7-81a2-8edf8673d1a1 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:07.378176 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd63b926-2187-4770-aa8a-4540ef687740 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:07.378851 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd63b926-2187-4770-aa8a-4540ef687740 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:07.379140 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd63b926-2187-4770-aa8a-4540ef687740 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:07.379425 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd63b926-2187-4770-aa8a-4540ef687740 None admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:07.380258 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd63b926-2187-4770-aa8a-4540ef687740 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:02:07.380574 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd63b926-2187-4770-aa8a-4540ef687740 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:07.382024 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd63b926-2187-4770-aa8a-4540ef687740 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:07.387843 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fb9a17-23d3-415a-9479-8ce122baa3da None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:07.395711 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 313/625] 10.222.0.42 () {56 vars in 1148 bytes} [Mon Jul 25 12:02:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:02:07.396552 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd63b926-2187-4770-aa8a-4540ef687740 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:07.405401 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:07.427893 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b68a66b-1ddb-4e95-bc2a-80c4723bca40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:07.428949 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b68a66b-1ddb-4e95-bc2a-80c4723bca40 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:07.429396 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b68a66b-1ddb-4e95-bc2a-80c4723bca40 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:07.429817 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b68a66b-1ddb-4e95-bc2a-80c4723bca40 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:07.441294 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b68a66b-1ddb-4e95-bc2a-80c4723bca40 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:07.443027 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b68a66b-1ddb-4e95-bc2a-80c4723bca40 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:07.462269 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-4b68a66b-1ddb-4e95-bc2a-80c4723bca40 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:02:07.462269 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:02:07.462269 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:02:07.462269 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:02:07.462269 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:02:07.462269 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:02:07.462269 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:02:07.462269 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:02:07.462269 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:02:07.462269 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:02:07.462269 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:02:07.462269 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:02:07.462269 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 12:02:07.462269 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 12:02:07.462269 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 12:02:07.463370 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 12:02:07.463370 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:02:07.463370 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:02:07.463370 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:02:07.463370 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:02:07.463370 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:02:07.463370 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:02:07.463370 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:02:07.463370 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jul 25 12:02:07.463370 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:02:07.463370 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:02:07.463370 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:02:07.463370 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:02:07.463370 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:02:07.463370 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:02:07.463370 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:02:07.463370 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:02:07.464451 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 12:02:07.464451 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 12:02:07.464451 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:02:07.464451 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jul 25 12:02:07.466269 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 313/626] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jul 25 12:02:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:02:07.467325 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b68a66b-1ddb-4e95-bc2a-80c4723bca40 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:07.471759 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:07.506392 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd63b926-2187-4770-aa8a-4540ef687740 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:07.544285 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a45894b-f43f-4ade-849d-9d10921b9eb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:07.545090 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a45894b-f43f-4ade-849d-9d10921b9eb2 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:07.545435 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a45894b-f43f-4ade-849d-9d10921b9eb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:07.545718 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a45894b-f43f-4ade-849d-9d10921b9eb2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:07.557117 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a45894b-f43f-4ade-849d-9d10921b9eb2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 12:02:07.565487 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-4a45894b-f43f-4ade-849d-9d10921b9eb2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 25 12:02:07.565487 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:02:07.565487 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:02:07.565487 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:02:07.565487 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:02:07.565487 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:02:07.565487 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:02:07.565487 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:02:07.565487 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:02:07.565487 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:02:07.565487 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:02:07.565487 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:02:07.565487 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 12:02:07.565487 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 12:02:07.565487 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 12:02:07.565487 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 12:02:07.566406 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 12:02:07.566406 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 12:02:07.566406 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 12:02:07.566406 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 12:02:07.566406 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:02:07.566406 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:02:07.566406 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:02:07.566406 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:02:07.566406 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:02:07.566406 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:02:07.566406 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:02:07.566406 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 25 12:02:07.566406 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:02:07.566406 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:02:07.566406 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:02:07.566406 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:02:07.567284 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:02:07.567284 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:02:07.567284 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:02:07.567284 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:02:07.567284 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 12:02:07.567284 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 12:02:07.567284 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 12:02:07.567284 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 12:02:07.567284 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 12:02:07.567284 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 12:02:07.567284 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 25 12:02:07.567284 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 25 12:02:07.571590 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 314/627] 10.222.0.42 () {56 vars in 1102 bytes} [Mon Jul 25 12:02:07 2022] GET /identity/v3/roles/service => generated 85 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:02:07.572420 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a45894b-f43f-4ade-849d-9d10921b9eb2 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:07.578448 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b68a66b-1ddb-4e95-bc2a-80c4723bca40 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:07.616251 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574edb18-194d-4866-add6-5388abfdd4c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:07.617248 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574edb18-194d-4866-add6-5388abfdd4c3 None admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:07.617539 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574edb18-194d-4866-add6-5388abfdd4c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:07.617815 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574edb18-194d-4866-add6-5388abfdd4c3 None admin] PATH_INFO: `/v3/projects/b9f62b8f37304b8a9a4bb54205347577/users/7bd782ea66364eeb96592096eae63d4f/roles/79dec36aee5b4322bf652041c25ca6e4` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:07.633156 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574edb18-194d-4866-add6-5388abfdd4c3 None admin] RBAC: Authorizing `identity:create_grant(project_id=b9f62b8f37304b8a9a4bb54205347577, user_id=7bd782ea66364eeb96592096eae63d4f, role_id=79dec36aee5b4322bf652041c25ca6e4)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:07.634726 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574edb18-194d-4866-add6-5388abfdd4c3 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:07.644344 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:07.672134 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 314/628] 10.222.0.42 () {58 vars in 1439 bytes} [Mon Jul 25 12:02:07 2022] PUT /identity/v3/projects/b9f62b8f37304b8a9a4bb54205347577/users/7bd782ea66364eeb96592096eae63d4f/roles/79dec36aee5b4322bf652041c25ca6e4 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:02:07.672134 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574edb18-194d-4866-add6-5388abfdd4c3 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:07.707937 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a45894b-f43f-4ade-849d-9d10921b9eb2 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:07.789132 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca72b90f-2140-4f58-90c4-d61ed9d10631 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:07.790164 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca72b90f-2140-4f58-90c4-d61ed9d10631 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:07.790658 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca72b90f-2140-4f58-90c4-d61ed9d10631 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:07.791409 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca72b90f-2140-4f58-90c4-d61ed9d10631 None admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:07.792374 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca72b90f-2140-4f58-90c4-d61ed9d10631 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:02:07.792938 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca72b90f-2140-4f58-90c4-d61ed9d10631 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:07.794652 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca72b90f-2140-4f58-90c4-d61ed9d10631 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:07.795958 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:07.823557 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 315/629] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jul 25 12:02:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:02:07.829215 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca72b90f-2140-4f58-90c4-d61ed9d10631 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:07.856307 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574edb18-194d-4866-add6-5388abfdd4c3 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:07.919695 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f6a34b-80b5-47e1-86b1-5c6ec4c1b0c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:07.920624 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f6a34b-80b5-47e1-86b1-5c6ec4c1b0c1 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:07.921234 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f6a34b-80b5-47e1-86b1-5c6ec4c1b0c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:07.921650 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f6a34b-80b5-47e1-86b1-5c6ec4c1b0c1 None admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:07.922545 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f6a34b-80b5-47e1-86b1-5c6ec4c1b0c1 None admin] RBAC: Adding query filter params (name=service) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:02:07.923004 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f6a34b-80b5-47e1-86b1-5c6ec4c1b0c1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:07.925169 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f6a34b-80b5-47e1-86b1-5c6ec4c1b0c1 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:07.934315 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 315/630] 10.222.0.42 () {56 vars in 1103 bytes} [Mon Jul 25 12:02:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:02:07.940064 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f6a34b-80b5-47e1-86b1-5c6ec4c1b0c1 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:07.991885 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca72b90f-2140-4f58-90c4-d61ed9d10631 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:08.022508 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:08.039209 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33f5f79-7e5d-4f3c-a669-be39e59f8adc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:08.040144 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33f5f79-7e5d-4f3c-a669-be39e59f8adc None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:08.040433 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33f5f79-7e5d-4f3c-a669-be39e59f8adc None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:08.040966 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33f5f79-7e5d-4f3c-a669-be39e59f8adc None admin] PATH_INFO: `/v3/users/glance` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:08.049829 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33f5f79-7e5d-4f3c-a669-be39e59f8adc None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:08.051851 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33f5f79-7e5d-4f3c-a669-be39e59f8adc None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:08.063572 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f6a34b-80b5-47e1-86b1-5c6ec4c1b0c1 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:08.069047 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-c33f5f79-7e5d-4f3c-a669-be39e59f8adc None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 25 12:02:08.069047 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:02:08.069047 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:02:08.069047 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:02:08.069047 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:02:08.069047 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:02:08.069047 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:02:08.069047 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:02:08.069047 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:02:08.069047 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:02:08.069047 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:02:08.069047 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:02:08.069047 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 12:02:08.069047 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 12:02:08.069047 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 12:02:08.069047 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 12:02:08.069984 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:02:08.069984 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:02:08.069984 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 12:02:08.069984 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 12:02:08.069984 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 12:02:08.069984 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 12:02:08.069984 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:02:08.069984 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:02:08.069984 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:02:08.069984 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:02:08.069984 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:02:08.069984 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 25 12:02:08.069984 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:02:08.069984 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:02:08.069984 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:02:08.069984 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:02:08.069984 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:02:08.069984 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:02:08.070922 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:02:08.070922 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:02:08.070922 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 12:02:08.070922 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 12:02:08.070922 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 12:02:08.070922 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 12:02:08.070922 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 12:02:08.070922 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 12:02:08.070922 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 25 12:02:08.070922 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 25 12:02:08.070922 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 316/631] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Jul 25 12:02:07 2022] GET /identity/v3/users/glance => generated 84 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 12:02:08.079565 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33f5f79-7e5d-4f3c-a669-be39e59f8adc None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:08.094347 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:08.130766 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3334de-34aa-40e7-a1cc-ca4a3a016625 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:08.131578 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3334de-34aa-40e7-a1cc-ca4a3a016625 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:08.131870 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3334de-34aa-40e7-a1cc-ca4a3a016625 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:08.132151 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3334de-34aa-40e7-a1cc-ca4a3a016625 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:08.156479 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3334de-34aa-40e7-a1cc-ca4a3a016625 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:08.157954 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3334de-34aa-40e7-a1cc-ca4a3a016625 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:08.180644 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-7b3334de-34aa-40e7-a1cc-ca4a3a016625 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:02:08.180644 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:02:08.180644 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:02:08.180644 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:02:08.180644 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:02:08.180644 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:02:08.180644 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:02:08.180644 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:02:08.180644 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:02:08.180644 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:02:08.180644 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:02:08.180644 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:02:08.180644 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 12:02:08.180644 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 12:02:08.180644 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 12:02:08.181581 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 12:02:08.181581 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:02:08.181581 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:02:08.181581 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:02:08.181581 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:02:08.181581 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:02:08.181581 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:02:08.181581 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:02:08.181581 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jul 25 12:02:08.181581 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:02:08.181581 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:02:08.181581 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:02:08.181581 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:02:08.181581 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:02:08.181581 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:02:08.181581 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:02:08.181581 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:02:08.182474 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 12:02:08.182474 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 12:02:08.182474 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:02:08.182474 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jul 25 12:02:08.184883 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 316/632] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jul 25 12:02:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 12:02:08.190419 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3334de-34aa-40e7-a1cc-ca4a3a016625 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:08.225572 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33f5f79-7e5d-4f3c-a669-be39e59f8adc None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:08.301341 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39197354-ab2f-4936-88b0-dbe821a48054 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:08.302504 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39197354-ab2f-4936-88b0-dbe821a48054 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:08.302936 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39197354-ab2f-4936-88b0-dbe821a48054 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:08.303642 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39197354-ab2f-4936-88b0-dbe821a48054 None admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:08.305053 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39197354-ab2f-4936-88b0-dbe821a48054 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:02:08.305742 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39197354-ab2f-4936-88b0-dbe821a48054 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:08.307964 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39197354-ab2f-4936-88b0-dbe821a48054 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:08.332077 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 317/633] 10.222.0.42 () {56 vars in 1137 bytes} [Mon Jul 25 12:02:08 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 12:02:08.334335 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabd1cea-7164-4695-aa2c-854dee7222fc None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:08.334796 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabd1cea-7164-4695-aa2c-854dee7222fc None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:08.335467 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabd1cea-7164-4695-aa2c-854dee7222fc None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:08.337551 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 318/634] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 25 12:02:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:02:08.338422 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabd1cea-7164-4695-aa2c-854dee7222fc None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:08.357096 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3334de-34aa-40e7-a1cc-ca4a3a016625 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:08.394747 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286b7cf4-92f4-4cf6-8c41-ee6860ea863c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:08.395607 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286b7cf4-92f4-4cf6-8c41-ee6860ea863c None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:08.395995 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286b7cf4-92f4-4cf6-8c41-ee6860ea863c None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:08.396396 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286b7cf4-92f4-4cf6-8c41-ee6860ea863c None admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:08.397201 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286b7cf4-92f4-4cf6-8c41-ee6860ea863c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:02:08.397587 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286b7cf4-92f4-4cf6-8c41-ee6860ea863c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:08.399216 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286b7cf4-92f4-4cf6-8c41-ee6860ea863c None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:08.403186 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:08.418956 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 317/635] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jul 25 12:02:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 12:02:08.418956 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aac6271-5f48-4bd7-907e-c70dc7dbf0bc None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:08.418956 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aac6271-5f48-4bd7-907e-c70dc7dbf0bc None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:08.418956 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aac6271-5f48-4bd7-907e-c70dc7dbf0bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:08.441119 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-8aac6271-5f48-4bd7-907e-c70dc7dbf0bc None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:02:08.442443 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-8aac6271-5f48-4bd7-907e-c70dc7dbf0bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:02:08.474189 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:08.477876 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabd1cea-7164-4695-aa2c-854dee7222fc None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:08.528473 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6b8408-742f-4a2a-8cf6-d9377b104606 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:08.529229 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6b8408-742f-4a2a-8cf6-d9377b104606 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:08.529579 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6b8408-742f-4a2a-8cf6-d9377b104606 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:08.529983 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6b8408-742f-4a2a-8cf6-d9377b104606 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:08.532566 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 318/636] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 25 12:02:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 12:02:08.534171 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aac6271-5f48-4bd7-907e-c70dc7dbf0bc None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:08.624814 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6b8408-742f-4a2a-8cf6-d9377b104606 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:08.626534 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6b8408-742f-4a2a-8cf6-d9377b104606 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:08.672551 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aac6271-5f48-4bd7-907e-c70dc7dbf0bc None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:08.673765 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:08.713994 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93991a92-83b2-4a7d-b4b9-59af5703cf69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:08.715140 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93991a92-83b2-4a7d-b4b9-59af5703cf69 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:08.715494 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93991a92-83b2-4a7d-b4b9-59af5703cf69 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:08.715894 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93991a92-83b2-4a7d-b4b9-59af5703cf69 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:08.726219 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93991a92-83b2-4a7d-b4b9-59af5703cf69 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:08.728304 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93991a92-83b2-4a7d-b4b9-59af5703cf69 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:08.732910 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 319/637] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Jul 25 12:02:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 12:02:08.733926 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6b8408-742f-4a2a-8cf6-d9377b104606 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:08.748217 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-93991a92-83b2-4a7d-b4b9-59af5703cf69 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 25 12:02:08.748217 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:02:08.748217 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:02:08.748217 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:02:08.748217 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:02:08.748217 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:02:08.748217 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:02:08.748217 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:02:08.748217 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:02:08.748217 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:02:08.748217 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:02:08.748217 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:02:08.748217 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 12:02:08.748217 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 12:02:08.748217 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 12:02:08.748217 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 12:02:08.749209 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:02:08.749209 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:02:08.749209 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 12:02:08.749209 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 12:02:08.749209 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 12:02:08.749209 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 12:02:08.749209 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:02:08.749209 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:02:08.749209 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:02:08.749209 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:02:08.749209 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:02:08.749209 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jul 25 12:02:08.749209 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:02:08.749209 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:02:08.749209 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:02:08.749209 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:02:08.749209 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:02:08.749209 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:02:08.750114 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:02:08.750114 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:02:08.750114 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 12:02:08.750114 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 12:02:08.750114 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 12:02:08.750114 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 12:02:08.750114 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 12:02:08.750114 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 12:02:08.750114 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 25 12:02:08.750114 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jul 25 12:02:08.750114 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 319/638] 10.222.0.42 () {56 vars in 1093 bytes} [Mon Jul 25 12:02:08 2022] GET /identity/v3/users/nova => generated 82 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:02:08.750685 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93991a92-83b2-4a7d-b4b9-59af5703cf69 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:08.773403 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:08.854454 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6b8408-742f-4a2a-8cf6-d9377b104606 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:08.857557 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93991a92-83b2-4a7d-b4b9-59af5703cf69 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:08.893674 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d545b4-d357-448c-9ec2-3874d2a3398f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:08.894390 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d545b4-d357-448c-9ec2-3874d2a3398f None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:08.894675 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d545b4-d357-448c-9ec2-3874d2a3398f None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:08.894972 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d545b4-d357-448c-9ec2-3874d2a3398f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:08.896574 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46982151-9aa0-4d98-9e4b-58b8c0cddefb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:08.897727 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46982151-9aa0-4d98-9e4b-58b8c0cddefb None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:08.898157 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46982151-9aa0-4d98-9e4b-58b8c0cddefb None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:08.898598 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46982151-9aa0-4d98-9e4b-58b8c0cddefb None admin] PATH_INFO: `/v3/roles/service` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:08.905063 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d545b4-d357-448c-9ec2-3874d2a3398f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:08.906556 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d545b4-d357-448c-9ec2-3874d2a3398f None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:08.910154 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46982151-9aa0-4d98-9e4b-58b8c0cddefb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 12:02:08.918468 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-46982151-9aa0-4d98-9e4b-58b8c0cddefb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 25 12:02:08.918468 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:02:08.918468 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:02:08.918468 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:02:08.918468 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:02:08.918468 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:02:08.918468 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:02:08.918468 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:02:08.918468 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:02:08.918468 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:02:08.918468 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:02:08.918468 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:02:08.918468 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 12:02:08.918468 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 12:02:08.918468 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 12:02:08.918468 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 12:02:08.919571 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 12:02:08.919571 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 12:02:08.919571 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 12:02:08.919571 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 12:02:08.919571 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:02:08.919571 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:02:08.919571 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:02:08.919571 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:02:08.919571 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:02:08.919571 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:02:08.919571 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:02:08.919571 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 25 12:02:08.919571 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:02:08.919571 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:02:08.919571 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:02:08.919571 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:02:08.920591 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:02:08.920591 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:02:08.920591 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:02:08.920591 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:02:08.920591 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 12:02:08.920591 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 12:02:08.920591 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 12:02:08.920591 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 12:02:08.920591 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 12:02:08.920591 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 12:02:08.920591 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 25 12:02:08.920591 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 25 12:02:08.922596 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 320/639] 10.222.0.42 () {56 vars in 1102 bytes} [Mon Jul 25 12:02:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:02:08.923978 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1104120-21b7-497a-83e2-7ef0891566d2 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:08.924262 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-94d545b4-d357-448c-9ec2-3874d2a3398f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:02:08.924262 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:02:08.924262 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:02:08.924262 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:02:08.924262 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:02:08.924262 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:02:08.924262 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:02:08.924262 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:02:08.924262 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:02:08.924262 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:02:08.924262 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:02:08.924262 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:02:08.924262 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 12:02:08.924262 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 12:02:08.924262 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 12:02:08.925153 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 12:02:08.925153 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:02:08.925153 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:02:08.925153 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:02:08.925153 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:02:08.925153 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:02:08.925153 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:02:08.925153 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:02:08.925153 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jul 25 12:02:08.925153 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:02:08.925153 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:02:08.925153 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:02:08.925153 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:02:08.925153 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:02:08.925153 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:02:08.925153 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:02:08.925153 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:02:08.930659 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 12:02:08.930659 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 12:02:08.930659 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:02:08.930659 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jul 25 12:02:08.930659 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 320/640] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jul 25 12:02:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:02:08.930659 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d545b4-d357-448c-9ec2-3874d2a3398f None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:08.931019 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1104120-21b7-497a-83e2-7ef0891566d2 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:08.931019 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1104120-21b7-497a-83e2-7ef0891566d2 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:08.931019 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 321/641] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 25 12:02:08 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:02:08.931019 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1104120-21b7-497a-83e2-7ef0891566d2 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:08.934867 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:09.002879 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:09.035723 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d545b4-d357-448c-9ec2-3874d2a3398f None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:09.039135 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1104120-21b7-497a-83e2-7ef0891566d2 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:09.072856 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba57efb-5fe6-48f0-940d-de383f23ba62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:09.073527 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba57efb-5fe6-48f0-940d-de383f23ba62 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:09.073812 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba57efb-5fe6-48f0-940d-de383f23ba62 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:09.074090 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba57efb-5fe6-48f0-940d-de383f23ba62 None admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:09.074919 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba57efb-5fe6-48f0-940d-de383f23ba62 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:02:09.075229 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba57efb-5fe6-48f0-940d-de383f23ba62 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:09.076664 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba57efb-5fe6-48f0-940d-de383f23ba62 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:09.077478 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b050098-6812-491d-b643-131f774b0e71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:09.078426 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b050098-6812-491d-b643-131f774b0e71 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:09.078871 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b050098-6812-491d-b643-131f774b0e71 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:09.079379 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b050098-6812-491d-b643-131f774b0e71 None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:09.080300 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b050098-6812-491d-b643-131f774b0e71 None admin] RBAC: Adding query filter params (name=service) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:02:09.080618 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b050098-6812-491d-b643-131f774b0e71 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:09.082074 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b050098-6812-491d-b643-131f774b0e71 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:09.090728 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 322/642] 10.222.0.42 () {56 vars in 1103 bytes} [Mon Jul 25 12:02:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:02:09.090728 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b050098-6812-491d-b643-131f774b0e71 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:09.102170 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 321/643] 10.222.0.42 () {56 vars in 1133 bytes} [Mon Jul 25 12:02:08 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:02:09.102170 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82396cc0-5691-4a0d-95af-f887dbaabeb0 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:09.102170 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82396cc0-5691-4a0d-95af-f887dbaabeb0 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:09.102170 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82396cc0-5691-4a0d-95af-f887dbaabeb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:09.118118 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-82396cc0-5691-4a0d-95af-f887dbaabeb0 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:02:09.119315 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-82396cc0-5691-4a0d-95af-f887dbaabeb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:02:09.172816 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:09.194395 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 322/644] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 25 12:02:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 12:02:09.195264 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82396cc0-5691-4a0d-95af-f887dbaabeb0 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:09.219796 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b050098-6812-491d-b643-131f774b0e71 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:09.258961 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:09.259851 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ce809d-a0d5-48a9-b1e8-c12c31b14335 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:09.260524 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ce809d-a0d5-48a9-b1e8-c12c31b14335 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:09.260864 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ce809d-a0d5-48a9-b1e8-c12c31b14335 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:09.261195 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ce809d-a0d5-48a9-b1e8-c12c31b14335 None admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:09.261953 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ce809d-a0d5-48a9-b1e8-c12c31b14335 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:02:09.262279 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ce809d-a0d5-48a9-b1e8-c12c31b14335 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:09.263715 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ce809d-a0d5-48a9-b1e8-c12c31b14335 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:09.275700 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 323/645] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jul 25 12:02:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:02:09.276493 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ce809d-a0d5-48a9-b1e8-c12c31b14335 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:09.306699 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82396cc0-5691-4a0d-95af-f887dbaabeb0 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:09.342231 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed936c46-4e7d-45cf-a165-5c994eff022d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:09.342981 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed936c46-4e7d-45cf-a165-5c994eff022d None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:09.343272 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed936c46-4e7d-45cf-a165-5c994eff022d None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:09.343575 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed936c46-4e7d-45cf-a165-5c994eff022d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:09.353867 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed936c46-4e7d-45cf-a165-5c994eff022d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:09.355322 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed936c46-4e7d-45cf-a165-5c994eff022d None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:09.373514 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-ed936c46-4e7d-45cf-a165-5c994eff022d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:02:09.373514 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:02:09.373514 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:02:09.373514 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:02:09.373514 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:02:09.373514 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:02:09.373514 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:02:09.373514 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:02:09.373514 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:02:09.373514 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:02:09.373514 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:02:09.373514 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:02:09.373514 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 12:02:09.373514 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 12:02:09.373514 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 12:02:09.374427 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 12:02:09.374427 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:02:09.374427 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:02:09.374427 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:02:09.374427 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:02:09.374427 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:02:09.374427 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:02:09.374427 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:02:09.374427 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jul 25 12:02:09.374427 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:02:09.374427 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:02:09.374427 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:02:09.374427 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:02:09.374427 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:02:09.374427 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:02:09.374427 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:02:09.374427 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:02:09.375475 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 12:02:09.375475 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 12:02:09.375475 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:02:09.375475 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jul 25 12:02:09.375475 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 323/646] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jul 25 12:02:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:02:09.375475 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed936c46-4e7d-45cf-a165-5c994eff022d None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:09.389995 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ce809d-a0d5-48a9-b1e8-c12c31b14335 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:09.416976 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:09.426954 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaea3dca-2606-4da2-ac2f-049dbe9d08ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:09.427536 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaea3dca-2606-4da2-ac2f-049dbe9d08ac None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:09.427847 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaea3dca-2606-4da2-ac2f-049dbe9d08ac None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:09.428120 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaea3dca-2606-4da2-ac2f-049dbe9d08ac None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:09.487043 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed936c46-4e7d-45cf-a165-5c994eff022d None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:09.498698 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:09.521328 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaea3dca-2606-4da2-ac2f-049dbe9d08ac None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:09.522938 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaea3dca-2606-4da2-ac2f-049dbe9d08ac None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:09.526658 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5081e087-09cf-431f-84c6-e7cb5e03de6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:09.526658 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5081e087-09cf-431f-84c6-e7cb5e03de6a None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:09.526945 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5081e087-09cf-431f-84c6-e7cb5e03de6a None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:09.527227 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5081e087-09cf-431f-84c6-e7cb5e03de6a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:09.550064 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5081e087-09cf-431f-84c6-e7cb5e03de6a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 12:02:09.557865 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-5081e087-09cf-431f-84c6-e7cb5e03de6a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 25 12:02:09.557865 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:02:09.557865 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:02:09.557865 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:02:09.557865 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:02:09.557865 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:02:09.557865 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:02:09.557865 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:02:09.557865 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:02:09.557865 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:02:09.557865 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:02:09.557865 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:02:09.557865 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 12:02:09.557865 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 12:02:09.557865 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 12:02:09.557865 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 12:02:09.558999 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 12:02:09.558999 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 12:02:09.558999 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 12:02:09.558999 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 12:02:09.558999 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:02:09.558999 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:02:09.558999 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:02:09.558999 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:02:09.558999 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:02:09.558999 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:02:09.558999 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:02:09.558999 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jul 25 12:02:09.558999 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:02:09.558999 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:02:09.558999 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:02:09.558999 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:02:09.560064 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:02:09.560064 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:02:09.560064 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:02:09.560064 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:02:09.560064 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 12:02:09.560064 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 12:02:09.560064 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 12:02:09.560064 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 12:02:09.560064 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 12:02:09.560064 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 12:02:09.560064 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 25 12:02:09.560064 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jul 25 12:02:09.560064 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 324/647] 10.222.0.42 () {56 vars in 1102 bytes} [Mon Jul 25 12:02:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:02:09.560774 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5081e087-09cf-431f-84c6-e7cb5e03de6a None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:09.633168 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 324/648] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Jul 25 12:02:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 12:02:09.634097 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaea3dca-2606-4da2-ac2f-049dbe9d08ac None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:09.663177 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:09.674269 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5081e087-09cf-431f-84c6-e7cb5e03de6a None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:09.711363 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431b3077-0ae2-4fec-9d6c-9cff1f1f177e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:09.712122 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431b3077-0ae2-4fec-9d6c-9cff1f1f177e None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:09.712415 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431b3077-0ae2-4fec-9d6c-9cff1f1f177e None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:09.712702 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431b3077-0ae2-4fec-9d6c-9cff1f1f177e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:09.720733 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431b3077-0ae2-4fec-9d6c-9cff1f1f177e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:09.722196 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431b3077-0ae2-4fec-9d6c-9cff1f1f177e None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:09.736534 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-431b3077-0ae2-4fec-9d6c-9cff1f1f177e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 12:02:09.736534 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:02:09.736534 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:02:09.736534 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:02:09.736534 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:02:09.736534 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:02:09.736534 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:02:09.736534 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:02:09.736534 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:02:09.736534 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:02:09.736534 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:02:09.736534 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:02:09.736534 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 12:02:09.736534 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 12:02:09.736534 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 12:02:09.737651 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 12:02:09.737651 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:02:09.737651 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:02:09.737651 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:02:09.737651 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:02:09.737651 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:02:09.737651 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:02:09.737651 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:02:09.737651 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jul 25 12:02:09.737651 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:02:09.737651 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:02:09.737651 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:02:09.737651 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:02:09.737651 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:02:09.737651 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:02:09.737651 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:02:09.737651 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:02:09.738724 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 12:02:09.738724 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 12:02:09.738724 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 12:02:09.738724 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 12:02:09.738724 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 12:02:09.738724 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 12:02:09.738724 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 12:02:09.738724 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jul 25 12:02:09.738724 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 325/649] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Jul 25 12:02:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:02:09.739212 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431b3077-0ae2-4fec-9d6c-9cff1f1f177e None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:09.747927 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaea3dca-2606-4da2-ac2f-049dbe9d08ac None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:09.781359 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:09.787675 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04408e38-1558-4ffb-884f-ad320745c6bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:09.788382 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04408e38-1558-4ffb-884f-ad320745c6bf None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:09.788674 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04408e38-1558-4ffb-884f-ad320745c6bf None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:09.788956 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04408e38-1558-4ffb-884f-ad320745c6bf None admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:09.789698 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04408e38-1558-4ffb-884f-ad320745c6bf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:02:09.790011 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04408e38-1558-4ffb-884f-ad320745c6bf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:09.791690 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04408e38-1558-4ffb-884f-ad320745c6bf None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:09.808912 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 325/650] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jul 25 12:02:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:02:09.808912 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04408e38-1558-4ffb-884f-ad320745c6bf None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:09.862908 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431b3077-0ae2-4fec-9d6c-9cff1f1f177e None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:09.898575 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:09.906615 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e69531d-9c55-494a-bb7d-953c2121d76a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:09.907814 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e69531d-9c55-494a-bb7d-953c2121d76a None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:09.908274 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e69531d-9c55-494a-bb7d-953c2121d76a None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:09.908725 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e69531d-9c55-494a-bb7d-953c2121d76a None admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:09.910308 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e69531d-9c55-494a-bb7d-953c2121d76a None admin] RBAC: Adding query filter params (name=service) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:02:09.910772 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e69531d-9c55-494a-bb7d-953c2121d76a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:09.912586 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e69531d-9c55-494a-bb7d-953c2121d76a None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:09.921020 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04408e38-1558-4ffb-884f-ad320745c6bf None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:09.925001 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 326/651] 10.222.0.42 () {56 vars in 1103 bytes} [Mon Jul 25 12:02:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:02:09.925869 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e69531d-9c55-494a-bb7d-953c2121d76a None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:09.970323 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3b855b-d2f3-451b-aa23-1ba2122b0e35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:09.971073 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3b855b-d2f3-451b-aa23-1ba2122b0e35 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:09.971363 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3b855b-d2f3-451b-aa23-1ba2122b0e35 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:09.971673 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3b855b-d2f3-451b-aa23-1ba2122b0e35 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:09.982505 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3b855b-d2f3-451b-aa23-1ba2122b0e35 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:09.983999 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3b855b-d2f3-451b-aa23-1ba2122b0e35 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:10.001643 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-1e3b855b-d2f3-451b-aa23-1ba2122b0e35 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:02:10.001643 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:02:10.001643 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:02:10.001643 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:02:10.001643 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:02:10.001643 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:02:10.001643 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:02:10.001643 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:02:10.001643 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:02:10.001643 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:02:10.001643 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:02:10.001643 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:02:10.001643 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 12:02:10.001643 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 12:02:10.001643 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 12:02:10.003278 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 12:02:10.003278 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:02:10.003278 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:02:10.003278 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:02:10.003278 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:02:10.003278 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:02:10.003278 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:02:10.003278 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:02:10.003278 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 25 12:02:10.003278 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:02:10.003278 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:02:10.003278 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:02:10.003278 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:02:10.003278 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:02:10.003278 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:02:10.003278 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:02:10.003278 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:02:10.008957 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 12:02:10.008957 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 12:02:10.008957 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:02:10.008957 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 25 12:02:10.008957 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 326/652] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jul 25 12:02:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:02:10.008957 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3b855b-d2f3-451b-aa23-1ba2122b0e35 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:10.023907 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:10.033699 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e69531d-9c55-494a-bb7d-953c2121d76a None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:10.067991 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be76db01-84b2-45c7-89bb-98ace801bbe6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:10.068643 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be76db01-84b2-45c7-89bb-98ace801bbe6 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:10.068929 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be76db01-84b2-45c7-89bb-98ace801bbe6 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:10.069206 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be76db01-84b2-45c7-89bb-98ace801bbe6 None admin] PATH_INFO: `/v3/projects` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:10.069982 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be76db01-84b2-45c7-89bb-98ace801bbe6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:02:10.070288 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be76db01-84b2-45c7-89bb-98ace801bbe6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:10.071735 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be76db01-84b2-45c7-89bb-98ace801bbe6 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:10.083366 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 327/653] 10.222.0.42 () {56 vars in 1148 bytes} [Mon Jul 25 12:02:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:02:10.084177 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be76db01-84b2-45c7-89bb-98ace801bbe6 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:10.106459 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3b855b-d2f3-451b-aa23-1ba2122b0e35 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:10.141064 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa46600-4ac8-42e8-bfed-7c72bba786ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:10.141855 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa46600-4ac8-42e8-bfed-7c72bba786ea None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:10.142148 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa46600-4ac8-42e8-bfed-7c72bba786ea None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:10.142451 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa46600-4ac8-42e8-bfed-7c72bba786ea None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:10.149851 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa46600-4ac8-42e8-bfed-7c72bba786ea None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:10.151289 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa46600-4ac8-42e8-bfed-7c72bba786ea None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:10.153045 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:10.167499 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-6aa46600-4ac8-42e8-bfed-7c72bba786ea None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 25 12:02:10.167499 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:02:10.167499 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:02:10.167499 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:02:10.167499 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:02:10.167499 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:02:10.167499 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:02:10.167499 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:02:10.167499 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:02:10.167499 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:02:10.167499 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:02:10.167499 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:02:10.167499 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 12:02:10.167499 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 12:02:10.167499 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 12:02:10.167499 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 12:02:10.168515 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:02:10.168515 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:02:10.168515 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 12:02:10.168515 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 12:02:10.168515 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 12:02:10.168515 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 12:02:10.168515 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:02:10.168515 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:02:10.168515 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:02:10.168515 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:02:10.168515 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:02:10.168515 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 25 12:02:10.168515 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:02:10.168515 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:02:10.168515 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:02:10.168515 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:02:10.168515 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:02:10.168515 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:02:10.169460 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:02:10.169460 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:02:10.169460 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 12:02:10.169460 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 12:02:10.169460 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 12:02:10.169460 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 12:02:10.169460 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 12:02:10.169460 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 12:02:10.169460 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 25 12:02:10.169460 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 25 12:02:10.169460 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 327/654] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Jul 25 12:02:10 2022] GET /identity/v3/users/cinder => generated 84 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:02:10.170053 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba24ef07-989a-4ca5-91a8-f5ba30fd59b8 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:10.170627 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba24ef07-989a-4ca5-91a8-f5ba30fd59b8 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:10.170627 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba24ef07-989a-4ca5-91a8-f5ba30fd59b8 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:10.171824 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 328/655] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 25 12:02:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:02:10.176483 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba24ef07-989a-4ca5-91a8-f5ba30fd59b8 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:10.186319 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be76db01-84b2-45c7-89bb-98ace801bbe6 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:10.219922 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91ca07f-72c1-4ff1-a5c4-8fe162740f60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:10.221319 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91ca07f-72c1-4ff1-a5c4-8fe162740f60 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:10.221684 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91ca07f-72c1-4ff1-a5c4-8fe162740f60 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:10.222111 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91ca07f-72c1-4ff1-a5c4-8fe162740f60 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:10.233076 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91ca07f-72c1-4ff1-a5c4-8fe162740f60 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:10.234541 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91ca07f-72c1-4ff1-a5c4-8fe162740f60 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:10.253962 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-d91ca07f-72c1-4ff1-a5c4-8fe162740f60 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:02:10.253962 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:02:10.253962 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:02:10.253962 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:02:10.253962 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:02:10.253962 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:02:10.253962 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:02:10.253962 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:02:10.253962 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:02:10.253962 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:02:10.253962 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:02:10.253962 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:02:10.253962 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 12:02:10.253962 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 12:02:10.253962 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 12:02:10.255062 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 12:02:10.255062 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:02:10.255062 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:02:10.255062 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:02:10.255062 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:02:10.255062 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:02:10.255062 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:02:10.255062 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:02:10.255062 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jul 25 12:02:10.255062 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:02:10.255062 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:02:10.255062 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:02:10.255062 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:02:10.255062 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:02:10.255062 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:02:10.255062 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:02:10.255062 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:02:10.255997 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 12:02:10.255997 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 12:02:10.255997 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:02:10.255997 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jul 25 12:02:10.255997 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 328/656] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jul 25 12:02:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:02:10.255997 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91ca07f-72c1-4ff1-a5c4-8fe162740f60 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:10.259576 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:10.284100 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba24ef07-989a-4ca5-91a8-f5ba30fd59b8 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:10.319130 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c91a06-2563-4ed6-80a0-3ada6ccafb5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:10.319824 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c91a06-2563-4ed6-80a0-3ada6ccafb5d None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:10.320113 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c91a06-2563-4ed6-80a0-3ada6ccafb5d None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:10.320398 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c91a06-2563-4ed6-80a0-3ada6ccafb5d None admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:10.321179 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c91a06-2563-4ed6-80a0-3ada6ccafb5d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:02:10.321498 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c91a06-2563-4ed6-80a0-3ada6ccafb5d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:10.322889 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c91a06-2563-4ed6-80a0-3ada6ccafb5d None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:10.336651 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 329/657] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jul 25 12:02:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:02:10.338425 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c91a06-2563-4ed6-80a0-3ada6ccafb5d None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:10.357552 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91ca07f-72c1-4ff1-a5c4-8fe162740f60 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:10.394596 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76000c27-e052-451e-b388-714b2bdcebc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:10.395245 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76000c27-e052-451e-b388-714b2bdcebc6 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:10.395530 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76000c27-e052-451e-b388-714b2bdcebc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:10.395832 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76000c27-e052-451e-b388-714b2bdcebc6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:10.396654 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76000c27-e052-451e-b388-714b2bdcebc6 None admin] RBAC: Adding query filter params (role.id=79dec36aee5b4322bf652041c25ca6e4, scope.project.id=b9f62b8f37304b8a9a4bb54205347577, user.id=5ea16528b1374bae9cf54603f7907bfd) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:02:10.396970 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76000c27-e052-451e-b388-714b2bdcebc6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:10.398080 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:10.398388 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76000c27-e052-451e-b388-714b2bdcebc6 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:10.404354 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 329/658] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Jul 25 12:02:10 2022] GET /identity/v3/role_assignments?user.id=5ea16528b1374bae9cf54603f7907bfd&scope.project.id=b9f62b8f37304b8a9a4bb54205347577&role.id=79dec36aee5b4322bf652041c25ca6e4 => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:02:10.405517 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f68fa4-94aa-45dd-9495-510cca534d6d None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:10.405803 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f68fa4-94aa-45dd-9495-510cca534d6d None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:10.406083 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f68fa4-94aa-45dd-9495-510cca534d6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:10.433461 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-f7f68fa4-94aa-45dd-9495-510cca534d6d None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:02:10.434741 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-f7f68fa4-94aa-45dd-9495-510cca534d6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:02:10.448230 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c91a06-2563-4ed6-80a0-3ada6ccafb5d None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:10.486984 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e926f09b-620e-4232-8b00-13f101e6bf36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:10.487619 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e926f09b-620e-4232-8b00-13f101e6bf36 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:10.487912 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e926f09b-620e-4232-8b00-13f101e6bf36 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:10.488203 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e926f09b-620e-4232-8b00-13f101e6bf36 None admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:10.489078 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e926f09b-620e-4232-8b00-13f101e6bf36 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:02:10.489399 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e926f09b-620e-4232-8b00-13f101e6bf36 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:10.490900 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e926f09b-620e-4232-8b00-13f101e6bf36 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:10.504827 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:10.515093 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 330/659] 10.222.0.42 () {56 vars in 1137 bytes} [Mon Jul 25 12:02:10 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:02:10.515093 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e926f09b-620e-4232-8b00-13f101e6bf36 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:10.519001 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 330/660] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 25 12:02:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 12:02:10.522811 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f68fa4-94aa-45dd-9495-510cca534d6d None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:10.636598 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e926f09b-620e-4232-8b00-13f101e6bf36 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:10.647386 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f68fa4-94aa-45dd-9495-510cca534d6d None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:10.656549 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:10.678499 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f6bc0c-603f-4bbd-b810-cf76e8cf26b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:10.679251 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f6bc0c-603f-4bbd-b810-cf76e8cf26b8 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:10.679765 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f6bc0c-603f-4bbd-b810-cf76e8cf26b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:10.680025 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f6bc0c-603f-4bbd-b810-cf76e8cf26b8 None admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:10.680820 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f6bc0c-603f-4bbd-b810-cf76e8cf26b8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:02:10.681195 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f6bc0c-603f-4bbd-b810-cf76e8cf26b8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:10.682674 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f6bc0c-603f-4bbd-b810-cf76e8cf26b8 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:10.684605 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e539f23-61b4-4b06-8e24-85ae4d733003 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:10.685474 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e539f23-61b4-4b06-8e24-85ae4d733003 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:10.685757 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e539f23-61b4-4b06-8e24-85ae4d733003 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:10.686032 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e539f23-61b4-4b06-8e24-85ae4d733003 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:10.692961 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e539f23-61b4-4b06-8e24-85ae4d733003 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:10.694381 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e539f23-61b4-4b06-8e24-85ae4d733003 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:10.695894 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 331/661] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jul 25 12:02:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:02:10.697354 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f6bc0c-603f-4bbd-b810-cf76e8cf26b8 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:10.708976 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-2e539f23-61b4-4b06-8e24-85ae4d733003 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 12:02:10.708976 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:02:10.708976 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:02:10.708976 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:02:10.708976 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:02:10.708976 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:02:10.708976 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:02:10.708976 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:02:10.708976 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:02:10.708976 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:02:10.708976 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:02:10.708976 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:02:10.708976 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 12:02:10.708976 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 12:02:10.708976 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 12:02:10.709984 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 12:02:10.709984 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:02:10.709984 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:02:10.709984 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:02:10.709984 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:02:10.709984 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:02:10.709984 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:02:10.709984 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:02:10.709984 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jul 25 12:02:10.709984 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:02:10.709984 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:02:10.709984 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:02:10.709984 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:02:10.709984 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:02:10.709984 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:02:10.709984 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:02:10.709984 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:02:10.715982 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 12:02:10.715982 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 12:02:10.715982 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 12:02:10.715982 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 12:02:10.715982 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 12:02:10.715982 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 12:02:10.715982 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 12:02:10.715982 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jul 25 12:02:10.715982 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 331/662] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Jul 25 12:02:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:02:10.715982 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e539f23-61b4-4b06-8e24-85ae4d733003 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:10.781225 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:10.818643 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f6bc0c-603f-4bbd-b810-cf76e8cf26b8 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:10.826920 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e539f23-61b4-4b06-8e24-85ae4d733003 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:10.856606 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7b33a3-21fc-4b10-9304-5a0182586c1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:10.857402 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7b33a3-21fc-4b10-9304-5a0182586c1c None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:10.857926 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7b33a3-21fc-4b10-9304-5a0182586c1c None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:10.858605 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7b33a3-21fc-4b10-9304-5a0182586c1c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:10.863141 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daebd5d0-8b4c-49c3-8fc8-2b083178803b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:10.863971 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daebd5d0-8b4c-49c3-8fc8-2b083178803b None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:10.864259 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daebd5d0-8b4c-49c3-8fc8-2b083178803b None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:10.864542 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daebd5d0-8b4c-49c3-8fc8-2b083178803b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:10.875126 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daebd5d0-8b4c-49c3-8fc8-2b083178803b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 12:02:10.883405 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-daebd5d0-8b4c-49c3-8fc8-2b083178803b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 25 12:02:10.883405 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:02:10.883405 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:02:10.883405 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:02:10.883405 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:02:10.883405 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:02:10.883405 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:02:10.883405 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:02:10.883405 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:02:10.883405 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:02:10.883405 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:02:10.883405 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:02:10.883405 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 12:02:10.883405 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 12:02:10.883405 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 12:02:10.883405 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 12:02:10.884492 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 12:02:10.884492 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 12:02:10.884492 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 12:02:10.884492 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 12:02:10.884492 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:02:10.884492 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:02:10.884492 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:02:10.884492 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:02:10.884492 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:02:10.884492 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:02:10.884492 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:02:10.884492 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jul 25 12:02:10.884492 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:02:10.884492 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:02:10.884492 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:02:10.884492 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:02:10.885577 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:02:10.885577 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:02:10.885577 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:02:10.885577 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:02:10.885577 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 12:02:10.885577 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 12:02:10.885577 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 12:02:10.885577 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 12:02:10.885577 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 12:02:10.885577 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 12:02:10.885577 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 25 12:02:10.885577 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jul 25 12:02:10.885577 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 332/663] 10.222.0.42 () {56 vars in 1102 bytes} [Mon Jul 25 12:02:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:02:10.886379 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daebd5d0-8b4c-49c3-8fc8-2b083178803b None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:10.915157 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:10.942116 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7b33a3-21fc-4b10-9304-5a0182586c1c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:10.943586 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7b33a3-21fc-4b10-9304-5a0182586c1c None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:10.999594 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daebd5d0-8b4c-49c3-8fc8-2b083178803b None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:11.017745 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:11.036934 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd1988c-5c5c-4d34-bf31-7ef49ec81a22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:11.037710 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd1988c-5c5c-4d34-bf31-7ef49ec81a22 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:11.038008 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd1988c-5c5c-4d34-bf31-7ef49ec81a22 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:11.038286 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd1988c-5c5c-4d34-bf31-7ef49ec81a22 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:11.046789 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd1988c-5c5c-4d34-bf31-7ef49ec81a22 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:11.048237 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd1988c-5c5c-4d34-bf31-7ef49ec81a22 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:11.060748 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 332/664] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Jul 25 12:02:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 12:02:11.060748 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7b33a3-21fc-4b10-9304-5a0182586c1c None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:11.063849 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-7cd1988c-5c5c-4d34-bf31-7ef49ec81a22 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 25 12:02:11.063849 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:02:11.063849 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:02:11.063849 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:02:11.063849 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:02:11.063849 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:02:11.063849 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:02:11.063849 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:02:11.063849 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:02:11.063849 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:02:11.063849 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:02:11.063849 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:02:11.063849 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 12:02:11.063849 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 12:02:11.063849 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 12:02:11.063849 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 12:02:11.064823 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:02:11.064823 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:02:11.064823 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 12:02:11.064823 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 12:02:11.064823 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 12:02:11.064823 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 12:02:11.064823 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:02:11.064823 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:02:11.064823 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:02:11.064823 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:02:11.064823 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:02:11.064823 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jul 25 12:02:11.064823 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:02:11.064823 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:02:11.064823 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:02:11.064823 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:02:11.064823 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:02:11.064823 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:02:11.065755 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:02:11.065755 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:02:11.065755 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 12:02:11.065755 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 12:02:11.065755 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 12:02:11.065755 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 12:02:11.065755 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 12:02:11.065755 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 12:02:11.065755 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 25 12:02:11.065755 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jul 25 12:02:11.065755 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 333/665] 10.222.0.42 () {56 vars in 1102 bytes} [Mon Jul 25 12:02:10 2022] GET /identity/v3/users/neutron => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:02:11.066354 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd1988c-5c5c-4d34-bf31-7ef49ec81a22 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:11.150093 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:11.177007 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7b33a3-21fc-4b10-9304-5a0182586c1c None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:11.177683 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd1988c-5c5c-4d34-bf31-7ef49ec81a22 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:11.212651 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1000fbdf-1ef5-412c-8fa1-e7b1adef2077 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:11.213317 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1000fbdf-1ef5-412c-8fa1-e7b1adef2077 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:11.213599 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1000fbdf-1ef5-412c-8fa1-e7b1adef2077 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:11.213875 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1000fbdf-1ef5-412c-8fa1-e7b1adef2077 None admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:11.214660 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1000fbdf-1ef5-412c-8fa1-e7b1adef2077 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:02:11.214969 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1000fbdf-1ef5-412c-8fa1-e7b1adef2077 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:11.215524 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88f52b5-de33-469a-8522-f73b6e26b286 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:11.216249 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88f52b5-de33-469a-8522-f73b6e26b286 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:11.216483 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1000fbdf-1ef5-412c-8fa1-e7b1adef2077 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:11.216590 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88f52b5-de33-469a-8522-f73b6e26b286 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:11.216816 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88f52b5-de33-469a-8522-f73b6e26b286 None admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:11.217566 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88f52b5-de33-469a-8522-f73b6e26b286 None admin] RBAC: Adding query filter params (name=service) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:02:11.217884 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88f52b5-de33-469a-8522-f73b6e26b286 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:11.219233 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88f52b5-de33-469a-8522-f73b6e26b286 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:11.232756 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 334/666] 10.222.0.42 () {56 vars in 1103 bytes} [Mon Jul 25 12:02:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:02:11.232756 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88f52b5-de33-469a-8522-f73b6e26b286 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:11.232990 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 333/667] 10.222.0.42 () {56 vars in 1148 bytes} [Mon Jul 25 12:02:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:02:11.232990 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1000fbdf-1ef5-412c-8fa1-e7b1adef2077 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:11.278524 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:11.336944 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1000fbdf-1ef5-412c-8fa1-e7b1adef2077 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:11.339759 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88f52b5-de33-469a-8522-f73b6e26b286 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:11.375108 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1ed2eb-f5d2-4589-b711-d9d82198cb43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:11.375576 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9186506c-686c-4f18-a0db-89926e9ff4e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:11.375803 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1ed2eb-f5d2-4589-b711-d9d82198cb43 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:11.376096 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1ed2eb-f5d2-4589-b711-d9d82198cb43 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:11.376352 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9186506c-686c-4f18-a0db-89926e9ff4e6 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:11.376451 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1ed2eb-f5d2-4589-b711-d9d82198cb43 None admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:11.376635 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9186506c-686c-4f18-a0db-89926e9ff4e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:11.377522 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9186506c-686c-4f18-a0db-89926e9ff4e6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:11.377759 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1ed2eb-f5d2-4589-b711-d9d82198cb43 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:02:11.378075 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1ed2eb-f5d2-4589-b711-d9d82198cb43 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:11.379504 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1ed2eb-f5d2-4589-b711-d9d82198cb43 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:11.382037 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:11.389550 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9186506c-686c-4f18-a0db-89926e9ff4e6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:11.390969 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9186506c-686c-4f18-a0db-89926e9ff4e6 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:11.400466 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 334/668] 10.222.0.42 () {56 vars in 1139 bytes} [Mon Jul 25 12:02:11 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:02:11.401423 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1ed2eb-f5d2-4589-b711-d9d82198cb43 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:11.413524 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-9186506c-686c-4f18-a0db-89926e9ff4e6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:02:11.413524 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:02:11.413524 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:02:11.413524 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:02:11.413524 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:02:11.413524 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:02:11.413524 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:02:11.413524 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:02:11.413524 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:02:11.413524 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:02:11.413524 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:02:11.413524 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:02:11.413524 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 12:02:11.413524 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 12:02:11.413524 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 12:02:11.414471 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 12:02:11.414471 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:02:11.414471 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:02:11.414471 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:02:11.414471 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:02:11.414471 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:02:11.414471 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:02:11.414471 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:02:11.414471 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jul 25 12:02:11.414471 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:02:11.414471 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:02:11.414471 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:02:11.414471 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:02:11.414471 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:02:11.414471 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:02:11.414471 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:02:11.414471 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:02:11.415364 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 12:02:11.415364 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 12:02:11.415364 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:02:11.415364 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jul 25 12:02:11.415364 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 335/669] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jul 25 12:02:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:02:11.415951 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9186506c-686c-4f18-a0db-89926e9ff4e6 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:11.513047 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1ed2eb-f5d2-4589-b711-d9d82198cb43 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:11.513600 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:11.524049 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9186506c-686c-4f18-a0db-89926e9ff4e6 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:11.553673 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605f2978-b48f-4fd4-94f2-67c5759d7c44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:11.554440 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605f2978-b48f-4fd4-94f2-67c5759d7c44 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:11.554931 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605f2978-b48f-4fd4-94f2-67c5759d7c44 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:11.555040 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605f2978-b48f-4fd4-94f2-67c5759d7c44 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:11.560613 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165772f8-bb9c-46c3-b586-9e0bc687e542 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:11.561287 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165772f8-bb9c-46c3-b586-9e0bc687e542 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:11.561571 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165772f8-bb9c-46c3-b586-9e0bc687e542 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:11.561854 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165772f8-bb9c-46c3-b586-9e0bc687e542 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:11.562706 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165772f8-bb9c-46c3-b586-9e0bc687e542 None admin] RBAC: Adding query filter params (role.id=79dec36aee5b4322bf652041c25ca6e4, scope.project.id=b9f62b8f37304b8a9a4bb54205347577, user.id=f67142a1193d47c7bd33d7040268233c) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:02:11.563023 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165772f8-bb9c-46c3-b586-9e0bc687e542 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:11.564574 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165772f8-bb9c-46c3-b586-9e0bc687e542 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:11.566257 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605f2978-b48f-4fd4-94f2-67c5759d7c44 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:11.567940 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605f2978-b48f-4fd4-94f2-67c5759d7c44 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:11.571275 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 336/670] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Jul 25 12:02:11 2022] GET /identity/v3/role_assignments?user.id=f67142a1193d47c7bd33d7040268233c&scope.project.id=b9f62b8f37304b8a9a4bb54205347577&role.id=79dec36aee5b4322bf652041c25ca6e4 => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:02:11.572514 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165772f8-bb9c-46c3-b586-9e0bc687e542 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:11.595238 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-605f2978-b48f-4fd4-94f2-67c5759d7c44 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:02:11.595238 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:02:11.595238 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:02:11.595238 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:02:11.595238 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:02:11.595238 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:02:11.595238 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:02:11.595238 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:02:11.595238 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:02:11.595238 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:02:11.595238 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:02:11.595238 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:02:11.595238 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 12:02:11.595238 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 12:02:11.595238 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 12:02:11.596785 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 12:02:11.596785 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:02:11.596785 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:02:11.596785 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:02:11.596785 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:02:11.596785 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:02:11.596785 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:02:11.596785 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:02:11.596785 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 25 12:02:11.596785 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:02:11.596785 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:02:11.596785 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:02:11.596785 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:02:11.596785 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:02:11.596785 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:02:11.596785 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:02:11.596785 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:02:11.597667 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 12:02:11.597667 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 12:02:11.597667 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:02:11.597667 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 25 12:02:11.602189 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 335/671] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jul 25 12:02:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:02:11.603724 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605f2978-b48f-4fd4-94f2-67c5759d7c44 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:11.669977 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:11.709430 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165772f8-bb9c-46c3-b586-9e0bc687e542 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:11.735026 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605f2978-b48f-4fd4-94f2-67c5759d7c44 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:11.757503 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ef90d0-4ab1-4ac2-9ae8-30007df8f443 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:11.758165 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ef90d0-4ab1-4ac2-9ae8-30007df8f443 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:11.758461 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ef90d0-4ab1-4ac2-9ae8-30007df8f443 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:11.758748 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ef90d0-4ab1-4ac2-9ae8-30007df8f443 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:11.786302 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12aebfd8-29fb-48af-9f12-844b0591b777 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:11.787499 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12aebfd8-29fb-48af-9f12-844b0591b777 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:11.787832 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12aebfd8-29fb-48af-9f12-844b0591b777 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:11.788122 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12aebfd8-29fb-48af-9f12-844b0591b777 None admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:11.790452 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12aebfd8-29fb-48af-9f12-844b0591b777 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:02:11.790452 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12aebfd8-29fb-48af-9f12-844b0591b777 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:11.791106 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12aebfd8-29fb-48af-9f12-844b0591b777 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:11.796601 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:11.803251 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 336/672] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jul 25 12:02:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:02:11.804444 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12aebfd8-29fb-48af-9f12-844b0591b777 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:11.867614 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ef90d0-4ab1-4ac2-9ae8-30007df8f443 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:11.869173 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ef90d0-4ab1-4ac2-9ae8-30007df8f443 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:11.927257 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12aebfd8-29fb-48af-9f12-844b0591b777 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:11.934832 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:11.964117 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3864e5ac-ddca-4c16-8f56-eeadcbf9f5aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:11.964968 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3864e5ac-ddca-4c16-8f56-eeadcbf9f5aa None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:11.965361 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3864e5ac-ddca-4c16-8f56-eeadcbf9f5aa None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:11.965747 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3864e5ac-ddca-4c16-8f56-eeadcbf9f5aa None admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:11.966579 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3864e5ac-ddca-4c16-8f56-eeadcbf9f5aa None admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:02:11.966994 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3864e5ac-ddca-4c16-8f56-eeadcbf9f5aa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:11.968501 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3864e5ac-ddca-4c16-8f56-eeadcbf9f5aa None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:11.982174 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 337/673] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jul 25 12:02:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:02:11.983265 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3864e5ac-ddca-4c16-8f56-eeadcbf9f5aa None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:11.985214 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 337/674] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Jul 25 12:02:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 12:02:11.991649 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ef90d0-4ab1-4ac2-9ae8-30007df8f443 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:12.067404 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:12.114524 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3864e5ac-ddca-4c16-8f56-eeadcbf9f5aa None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:12.116717 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ef90d0-4ab1-4ac2-9ae8-30007df8f443 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:12.150808 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e91c864-8637-4370-ae65-4ffb14858805 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:12.151718 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e91c864-8637-4370-ae65-4ffb14858805 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:12.152108 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e91c864-8637-4370-ae65-4ffb14858805 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:12.152496 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e91c864-8637-4370-ae65-4ffb14858805 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:12.157844 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1640125-91d9-4c4a-9752-dad1147762e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:12.158876 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1640125-91d9-4c4a-9752-dad1147762e1 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:12.159316 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1640125-91d9-4c4a-9752-dad1147762e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:12.159790 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1640125-91d9-4c4a-9752-dad1147762e1 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:12.160221 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e91c864-8637-4370-ae65-4ffb14858805 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:12.161768 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e91c864-8637-4370-ae65-4ffb14858805 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:12.168394 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1640125-91d9-4c4a-9752-dad1147762e1 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:12.170001 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1640125-91d9-4c4a-9752-dad1147762e1 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:12.176797 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-0e91c864-8637-4370-ae65-4ffb14858805 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 12:02:12.176797 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:02:12.176797 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:02:12.176797 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:02:12.176797 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:02:12.176797 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:02:12.176797 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:02:12.176797 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:02:12.176797 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:02:12.176797 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:02:12.176797 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:02:12.176797 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:02:12.176797 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 12:02:12.176797 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 12:02:12.176797 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 12:02:12.177801 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 12:02:12.177801 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:02:12.177801 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:02:12.177801 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:02:12.177801 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:02:12.177801 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:02:12.177801 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:02:12.177801 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:02:12.177801 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 25 12:02:12.177801 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:02:12.177801 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:02:12.177801 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:02:12.177801 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:02:12.177801 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:02:12.177801 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:02:12.177801 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:02:12.177801 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:02:12.178703 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 12:02:12.178703 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 12:02:12.178703 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 12:02:12.178703 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 12:02:12.178703 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 12:02:12.178703 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 12:02:12.178703 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 12:02:12.178703 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 25 12:02:12.179146 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 338/675] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Jul 25 12:02:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:02:12.180092 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e91c864-8637-4370-ae65-4ffb14858805 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:12.189372 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-c1640125-91d9-4c4a-9752-dad1147762e1 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 25 12:02:12.189372 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:02:12.189372 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:02:12.189372 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:02:12.189372 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:02:12.189372 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:02:12.189372 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:02:12.189372 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:02:12.189372 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:02:12.189372 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:02:12.189372 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:02:12.189372 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:02:12.189372 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 12:02:12.189372 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 12:02:12.189372 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 12:02:12.189372 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 12:02:12.190351 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:02:12.190351 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:02:12.190351 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 12:02:12.190351 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 12:02:12.190351 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 12:02:12.190351 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 12:02:12.190351 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:02:12.190351 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:02:12.190351 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:02:12.190351 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:02:12.190351 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:02:12.190351 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jul 25 12:02:12.190351 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:02:12.190351 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:02:12.190351 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:02:12.190351 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:02:12.190351 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:02:12.190351 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:02:12.191285 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:02:12.191285 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:02:12.191285 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 12:02:12.191285 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 12:02:12.191285 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 12:02:12.191285 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 12:02:12.191285 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 12:02:12.191285 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 12:02:12.191285 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 25 12:02:12.191285 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jul 25 12:02:12.194161 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 338/676] 10.222.0.42 () {56 vars in 1096 bytes} [Mon Jul 25 12:02:11 2022] GET /identity/v3/users/swift => generated 83 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:02:12.195225 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1640125-91d9-4c4a-9752-dad1147762e1 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:12.210360 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:12.292115 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e91c864-8637-4370-ae65-4ffb14858805 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:12.319408 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1640125-91d9-4c4a-9752-dad1147762e1 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:12.342374 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26857608-0d2e-430c-9aa5-a12989c16eef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:12.343186 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26857608-0d2e-430c-9aa5-a12989c16eef None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:12.344893 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26857608-0d2e-430c-9aa5-a12989c16eef None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:12.347436 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26857608-0d2e-430c-9aa5-a12989c16eef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:12.360155 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26857608-0d2e-430c-9aa5-a12989c16eef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:12.361691 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26857608-0d2e-430c-9aa5-a12989c16eef None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:12.363337 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a11227f-a6ad-4a4d-9865-f1aead20789d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:12.364098 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a11227f-a6ad-4a4d-9865-f1aead20789d None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:12.364484 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a11227f-a6ad-4a4d-9865-f1aead20789d None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:12.364771 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a11227f-a6ad-4a4d-9865-f1aead20789d None admin] PATH_INFO: `/v3/projects` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:12.365567 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a11227f-a6ad-4a4d-9865-f1aead20789d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:02:12.365880 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a11227f-a6ad-4a4d-9865-f1aead20789d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:12.367419 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a11227f-a6ad-4a4d-9865-f1aead20789d None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:12.375341 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:12.380075 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-26857608-0d2e-430c-9aa5-a12989c16eef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:02:12.380075 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:02:12.380075 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:02:12.380075 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:02:12.380075 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:02:12.380075 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:02:12.380075 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:02:12.380075 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:02:12.380075 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:02:12.380075 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:02:12.380075 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:02:12.380075 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:02:12.380075 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 12:02:12.380075 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 12:02:12.380075 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 12:02:12.380967 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 12:02:12.380967 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:02:12.380967 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:02:12.380967 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:02:12.380967 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:02:12.380967 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:02:12.380967 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:02:12.380967 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:02:12.380967 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 25 12:02:12.380967 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:02:12.380967 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:02:12.380967 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:02:12.380967 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:02:12.380967 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:02:12.380967 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:02:12.380967 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:02:12.380967 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:02:12.381855 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 12:02:12.381855 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 12:02:12.381855 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:02:12.381855 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 25 12:02:12.385366 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 339/677] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jul 25 12:02:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:02:12.385482 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 339/678] 10.222.0.42 () {56 vars in 1148 bytes} [Mon Jul 25 12:02:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:02:12.386144 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26857608-0d2e-430c-9aa5-a12989c16eef None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:12.387634 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a11227f-a6ad-4a4d-9865-f1aead20789d None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:12.468991 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:12.491577 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26857608-0d2e-430c-9aa5-a12989c16eef None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:12.494474 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a11227f-a6ad-4a4d-9865-f1aead20789d None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:12.527562 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e00c77-cc44-43ec-94d9-35865f0ccbab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:12.528254 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e00c77-cc44-43ec-94d9-35865f0ccbab None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:12.528540 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e00c77-cc44-43ec-94d9-35865f0ccbab None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:12.528819 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e00c77-cc44-43ec-94d9-35865f0ccbab None admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:12.529663 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e00c77-cc44-43ec-94d9-35865f0ccbab None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:02:12.529972 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e00c77-cc44-43ec-94d9-35865f0ccbab None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:12.531301 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17a8341-7830-4eac-b2ad-914bf772282e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:12.531709 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e00c77-cc44-43ec-94d9-35865f0ccbab None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:12.531986 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17a8341-7830-4eac-b2ad-914bf772282e None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:12.532271 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17a8341-7830-4eac-b2ad-914bf772282e None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:12.532549 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17a8341-7830-4eac-b2ad-914bf772282e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:12.533574 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17a8341-7830-4eac-b2ad-914bf772282e None admin] RBAC: Adding query filter params (role.id=79dec36aee5b4322bf652041c25ca6e4, scope.project.id=b9f62b8f37304b8a9a4bb54205347577, user.id=6f89ce6f0cf04373a414d927fa0f8a7f) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:02:12.534632 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17a8341-7830-4eac-b2ad-914bf772282e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:12.536099 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17a8341-7830-4eac-b2ad-914bf772282e None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:12.551968 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 340/679] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Jul 25 12:02:12 2022] GET /identity/v3/role_assignments?user.id=6f89ce6f0cf04373a414d927fa0f8a7f&scope.project.id=b9f62b8f37304b8a9a4bb54205347577&role.id=79dec36aee5b4322bf652041c25ca6e4 => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:02:12.551968 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17a8341-7830-4eac-b2ad-914bf772282e None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:12.561496 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 340/680] 10.222.0.42 () {56 vars in 1135 bytes} [Mon Jul 25 12:02:12 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:02:12.563146 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6844711b-d00e-41e4-ad57-eb406d050c15 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:12.563513 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6844711b-d00e-41e4-ad57-eb406d050c15 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:12.563893 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6844711b-d00e-41e4-ad57-eb406d050c15 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:12.568534 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 341/681] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 25 12:02:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:02:12.568534 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6844711b-d00e-41e4-ad57-eb406d050c15 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:12.605613 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:12.678647 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17a8341-7830-4eac-b2ad-914bf772282e None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:12.701460 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6844711b-d00e-41e4-ad57-eb406d050c15 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:12.719248 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89c65f9-7f53-4271-bd09-671549034fec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:12.719883 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89c65f9-7f53-4271-bd09-671549034fec None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:12.720176 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89c65f9-7f53-4271-bd09-671549034fec None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:12.720504 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89c65f9-7f53-4271-bd09-671549034fec None admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:12.721235 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89c65f9-7f53-4271-bd09-671549034fec None admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:02:12.721551 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89c65f9-7f53-4271-bd09-671549034fec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:12.722922 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89c65f9-7f53-4271-bd09-671549034fec None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:12.736930 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 341/682] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jul 25 12:02:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:02:12.738385 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6110d46-605e-403d-bb52-92803a1d8c54 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:12.738760 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6110d46-605e-403d-bb52-92803a1d8c54 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:12.739133 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6110d46-605e-403d-bb52-92803a1d8c54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:12.743061 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0ec001-abab-4478-8de9-054c83cf5a54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:12.743773 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0ec001-abab-4478-8de9-054c83cf5a54 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:12.744094 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0ec001-abab-4478-8de9-054c83cf5a54 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:12.744442 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0ec001-abab-4478-8de9-054c83cf5a54 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:12.750020 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:12.760766 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-f6110d46-605e-403d-bb52-92803a1d8c54 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:02:12.762279 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-f6110d46-605e-403d-bb52-92803a1d8c54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:02:12.848724 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0ec001-abab-4478-8de9-054c83cf5a54 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:12.851243 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0ec001-abab-4478-8de9-054c83cf5a54 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:12.857418 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 342/683] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 25 12:02:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 12:02:12.863568 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6110d46-605e-403d-bb52-92803a1d8c54 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:12.906028 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:12.993343 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 342/684] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Jul 25 12:02:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 12:02:12.994142 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0ec001-abab-4478-8de9-054c83cf5a54 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:12.996399 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6110d46-605e-403d-bb52-92803a1d8c54 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:13.035875 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:13.039609 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8912aa34-522c-4ef6-816a-6eee449ec3c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:13.041005 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8912aa34-522c-4ef6-816a-6eee449ec3c3 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:13.041508 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8912aa34-522c-4ef6-816a-6eee449ec3c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:13.042467 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8912aa34-522c-4ef6-816a-6eee449ec3c3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:13.052326 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8912aa34-522c-4ef6-816a-6eee449ec3c3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:13.054503 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8912aa34-522c-4ef6-816a-6eee449ec3c3 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:13.072635 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-8912aa34-522c-4ef6-816a-6eee449ec3c3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 12:02:13.072635 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:02:13.072635 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:02:13.072635 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:02:13.072635 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:02:13.072635 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:02:13.072635 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:02:13.072635 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:02:13.072635 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:02:13.072635 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:02:13.072635 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:02:13.072635 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:02:13.072635 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 12:02:13.072635 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 12:02:13.072635 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 12:02:13.073604 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 12:02:13.073604 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:02:13.073604 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:02:13.073604 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:02:13.073604 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:02:13.073604 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:02:13.073604 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:02:13.073604 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:02:13.073604 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jul 25 12:02:13.073604 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:02:13.073604 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:02:13.073604 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:02:13.073604 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:02:13.073604 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:02:13.073604 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:02:13.073604 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:02:13.073604 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:02:13.074555 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 12:02:13.074555 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 12:02:13.074555 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 12:02:13.074555 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 12:02:13.074555 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 12:02:13.074555 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 12:02:13.074555 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 12:02:13.074555 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jul 25 12:02:13.076896 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 343/685] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Jul 25 12:02:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:02:13.078758 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8912aa34-522c-4ef6-816a-6eee449ec3c3 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:13.121386 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0ec001-abab-4478-8de9-054c83cf5a54 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:13.169299 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cccb95-6b80-4746-8440-23ea78b89814 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:13.170669 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cccb95-6b80-4746-8440-23ea78b89814 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:13.171078 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cccb95-6b80-4746-8440-23ea78b89814 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:13.171705 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cccb95-6b80-4746-8440-23ea78b89814 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:13.184281 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:13.186837 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27cccb95-6b80-4746-8440-23ea78b89814 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:13.188665 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27cccb95-6b80-4746-8440-23ea78b89814 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:13.196755 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8912aa34-522c-4ef6-816a-6eee449ec3c3 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:13.216007 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-27cccb95-6b80-4746-8440-23ea78b89814 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:02:13.216007 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:02:13.216007 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:02:13.216007 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:02:13.216007 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:02:13.216007 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:02:13.216007 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:02:13.216007 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:02:13.216007 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:02:13.216007 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:02:13.216007 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:02:13.216007 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:02:13.216007 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 12:02:13.216007 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 12:02:13.216007 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 12:02:13.216961 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 12:02:13.216961 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:02:13.216961 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:02:13.216961 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:02:13.216961 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:02:13.216961 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:02:13.216961 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:02:13.216961 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:02:13.216961 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 25 12:02:13.216961 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:02:13.216961 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:02:13.216961 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:02:13.216961 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:02:13.216961 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:02:13.216961 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:02:13.216961 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:02:13.216961 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:02:13.217850 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 12:02:13.217850 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 12:02:13.217850 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:02:13.217850 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 25 12:02:13.223309 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 343/686] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jul 25 12:02:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:02:13.224494 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cccb95-6b80-4746-8440-23ea78b89814 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:13.238706 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294d2961-e708-4651-b6d1-7d9190d9ce93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:13.239843 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294d2961-e708-4651-b6d1-7d9190d9ce93 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:13.240337 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294d2961-e708-4651-b6d1-7d9190d9ce93 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:13.240823 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294d2961-e708-4651-b6d1-7d9190d9ce93 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:13.252331 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294d2961-e708-4651-b6d1-7d9190d9ce93 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:13.254014 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294d2961-e708-4651-b6d1-7d9190d9ce93 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:13.273567 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-294d2961-e708-4651-b6d1-7d9190d9ce93 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:02:13.273567 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:02:13.273567 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:02:13.273567 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:02:13.273567 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:02:13.273567 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:02:13.273567 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:02:13.273567 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:02:13.273567 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:02:13.273567 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:02:13.273567 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:02:13.273567 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:02:13.273567 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 12:02:13.273567 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 12:02:13.273567 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 12:02:13.274525 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 12:02:13.274525 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:02:13.274525 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:02:13.274525 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:02:13.274525 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:02:13.274525 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:02:13.274525 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:02:13.274525 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:02:13.274525 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jul 25 12:02:13.274525 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:02:13.274525 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:02:13.274525 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:02:13.274525 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:02:13.274525 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:02:13.274525 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:02:13.274525 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:02:13.274525 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:02:13.275410 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 12:02:13.275410 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 12:02:13.275410 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:02:13.275410 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jul 25 12:02:13.277777 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 344/687] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jul 25 12:02:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:02:13.278841 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294d2961-e708-4651-b6d1-7d9190d9ce93 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:13.325872 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:13.339783 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cccb95-6b80-4746-8440-23ea78b89814 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:13.377045 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79217cae-d0ba-4de6-90ed-96a81a04f629 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:13.378066 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79217cae-d0ba-4de6-90ed-96a81a04f629 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:13.378557 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79217cae-d0ba-4de6-90ed-96a81a04f629 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:13.379050 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79217cae-d0ba-4de6-90ed-96a81a04f629 None admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:13.380077 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79217cae-d0ba-4de6-90ed-96a81a04f629 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:02:13.380596 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79217cae-d0ba-4de6-90ed-96a81a04f629 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:13.382276 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79217cae-d0ba-4de6-90ed-96a81a04f629 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:13.395040 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 344/688] 10.222.0.42 () {56 vars in 1148 bytes} [Mon Jul 25 12:02:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:02:13.396272 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79217cae-d0ba-4de6-90ed-96a81a04f629 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:13.416900 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294d2961-e708-4651-b6d1-7d9190d9ce93 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:13.453640 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3c77a8-7c7d-496c-a90c-d9db1e7a0e6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:13.454652 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3c77a8-7c7d-496c-a90c-d9db1e7a0e6c None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:13.455172 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3c77a8-7c7d-496c-a90c-d9db1e7a0e6c None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:13.455715 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3c77a8-7c7d-496c-a90c-d9db1e7a0e6c None admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:13.456658 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3c77a8-7c7d-496c-a90c-d9db1e7a0e6c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:02:13.457178 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3c77a8-7c7d-496c-a90c-d9db1e7a0e6c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:13.458729 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3c77a8-7c7d-496c-a90c-d9db1e7a0e6c None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:13.476654 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 345/689] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jul 25 12:02:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:02:13.477836 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3c77a8-7c7d-496c-a90c-d9db1e7a0e6c None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:13.480927 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:13.524179 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79217cae-d0ba-4de6-90ed-96a81a04f629 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:13.571063 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03105114-7cfa-4206-a362-5b04f65e444c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:13.571748 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03105114-7cfa-4206-a362-5b04f65e444c None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:13.572046 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03105114-7cfa-4206-a362-5b04f65e444c None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:13.572332 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03105114-7cfa-4206-a362-5b04f65e444c None admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:13.573061 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03105114-7cfa-4206-a362-5b04f65e444c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:02:13.573375 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03105114-7cfa-4206-a362-5b04f65e444c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:13.574751 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03105114-7cfa-4206-a362-5b04f65e444c None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:13.588861 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3c77a8-7c7d-496c-a90c-d9db1e7a0e6c None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:13.592497 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 345/690] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jul 25 12:02:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:02:13.593314 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03105114-7cfa-4206-a362-5b04f65e444c None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:13.607365 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:13.628939 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c901db-9a6d-4a40-8909-625fd0b3e3b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:13.630091 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c901db-9a6d-4a40-8909-625fd0b3e3b1 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:13.630671 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c901db-9a6d-4a40-8909-625fd0b3e3b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:13.631148 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c901db-9a6d-4a40-8909-625fd0b3e3b1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:13.632324 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c901db-9a6d-4a40-8909-625fd0b3e3b1 None admin] RBAC: Adding query filter params (role.id=79dec36aee5b4322bf652041c25ca6e4, scope.project.id=b9f62b8f37304b8a9a4bb54205347577, user.id=51733781201243e8abf9eca39b207750) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:02:13.633044 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c901db-9a6d-4a40-8909-625fd0b3e3b1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:13.634726 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c901db-9a6d-4a40-8909-625fd0b3e3b1 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:13.641980 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 346/691] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Jul 25 12:02:13 2022] GET /identity/v3/role_assignments?user.id=51733781201243e8abf9eca39b207750&scope.project.id=b9f62b8f37304b8a9a4bb54205347577&role.id=79dec36aee5b4322bf652041c25ca6e4 => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:02:13.643795 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c901db-9a6d-4a40-8909-625fd0b3e3b1 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:13.723041 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:13.728103 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03105114-7cfa-4206-a362-5b04f65e444c None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:13.762748 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c901db-9a6d-4a40-8909-625fd0b3e3b1 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:13.780155 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3167bbe2-8536-4a6b-bf69-57245820c1ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:13.780787 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3167bbe2-8536-4a6b-bf69-57245820c1ca None admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:13.781086 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3167bbe2-8536-4a6b-bf69-57245820c1ca None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:13.781375 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3167bbe2-8536-4a6b-bf69-57245820c1ca None admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:13.782927 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3167bbe2-8536-4a6b-bf69-57245820c1ca None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:13.785672 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3167bbe2-8536-4a6b-bf69-57245820c1ca None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:13.827672 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 346/692] 10.222.0.42 () {60 vars in 1132 bytes} [Mon Jul 25 12:02:13 2022] POST /identity/v3/users => generated 254 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:02:13.828565 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20d48fe-8682-4ee2-9a14-2f3d19e98c7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:13.829678 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20d48fe-8682-4ee2-9a14-2f3d19e98c7e None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:13.830775 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20d48fe-8682-4ee2-9a14-2f3d19e98c7e None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:13.831258 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20d48fe-8682-4ee2-9a14-2f3d19e98c7e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:13.840301 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20d48fe-8682-4ee2-9a14-2f3d19e98c7e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:13.842384 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20d48fe-8682-4ee2-9a14-2f3d19e98c7e None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:13.861013 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-b20d48fe-8682-4ee2-9a14-2f3d19e98c7e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 12:02:13.861013 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:02:13.861013 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:02:13.861013 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:02:13.861013 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:02:13.861013 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:02:13.861013 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:02:13.861013 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:02:13.861013 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:02:13.861013 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:02:13.861013 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:02:13.861013 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:02:13.861013 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 12:02:13.861013 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 12:02:13.861013 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 12:02:13.862062 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 12:02:13.862062 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:02:13.862062 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:02:13.862062 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:02:13.862062 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:02:13.862062 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:02:13.862062 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:02:13.862062 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:02:13.862062 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jul 25 12:02:13.862062 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:02:13.862062 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:02:13.862062 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:02:13.862062 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:02:13.862062 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:02:13.862062 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:02:13.862062 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:02:13.862062 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:02:13.862964 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 12:02:13.862964 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 12:02:13.862964 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 12:02:13.862964 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 12:02:13.862964 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 12:02:13.862964 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 12:02:13.862964 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 12:02:13.862964 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jul 25 12:02:13.865781 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 347/693] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Jul 25 12:02:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:02:13.872709 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20d48fe-8682-4ee2-9a14-2f3d19e98c7e None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:13.873848 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e48abb8-11f9-4bc3-b1ee-1ab8f213ef6e None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:13.874432 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e48abb8-11f9-4bc3-b1ee-1ab8f213ef6e None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:13.874953 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e48abb8-11f9-4bc3-b1ee-1ab8f213ef6e None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:13.877444 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 347/694] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 25 12:02:13 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:02:13.888433 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b9ed46-665c-4e96-9e4d-02d6df460376 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:13.888724 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b9ed46-665c-4e96-9e4d-02d6df460376 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:13.889011 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b9ed46-665c-4e96-9e4d-02d6df460376 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:13.913305 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-08b9ed46-665c-4e96-9e4d-02d6df460376 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:02:13.915434 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-08b9ed46-665c-4e96-9e4d-02d6df460376 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:02:14.007735 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:14.019276 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20d48fe-8682-4ee2-9a14-2f3d19e98c7e None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:14.022342 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 348/695] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 25 12:02:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 25 12:02:14.053309 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:14.054645 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b9ed46-665c-4e96-9e4d-02d6df460376 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:14.063858 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccb0387-339b-4adc-834a-778d6119a800 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:14.064790 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccb0387-339b-4adc-834a-778d6119a800 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:14.065228 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccb0387-339b-4adc-834a-778d6119a800 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:14.065657 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccb0387-339b-4adc-834a-778d6119a800 None admin] PATH_INFO: `/v3/projects` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:14.066597 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fccb0387-339b-4adc-834a-778d6119a800 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:02:14.067063 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fccb0387-339b-4adc-834a-778d6119a800 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:14.069468 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fccb0387-339b-4adc-834a-778d6119a800 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:14.082157 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 348/696] 10.222.0.42 () {56 vars in 1148 bytes} [Mon Jul 25 12:02:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:02:14.087743 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccb0387-339b-4adc-834a-778d6119a800 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:14.168849 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b9ed46-665c-4e96-9e4d-02d6df460376 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:14.198893 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccb0387-339b-4adc-834a-778d6119a800 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:14.209170 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14222a12-b681-4e89-8a7e-caf7f23aeeb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:14.209986 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14222a12-b681-4e89-8a7e-caf7f23aeeb4 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:14.210278 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14222a12-b681-4e89-8a7e-caf7f23aeeb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:14.210564 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14222a12-b681-4e89-8a7e-caf7f23aeeb4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:14.223986 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14222a12-b681-4e89-8a7e-caf7f23aeeb4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 12:02:14.232320 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-14222a12-b681-4e89-8a7e-caf7f23aeeb4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 12:02:14.232320 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:02:14.232320 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:02:14.232320 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:02:14.232320 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:02:14.232320 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:02:14.232320 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:02:14.232320 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:02:14.232320 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:02:14.232320 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:02:14.232320 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:02:14.232320 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:02:14.232320 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 12:02:14.232320 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 12:02:14.232320 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 12:02:14.232320 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 12:02:14.233487 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 12:02:14.233487 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 12:02:14.233487 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 12:02:14.233487 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 12:02:14.233487 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:02:14.233487 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:02:14.233487 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:02:14.233487 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:02:14.233487 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:02:14.233487 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:02:14.233487 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:02:14.233487 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 25 12:02:14.233487 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:02:14.233487 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:02:14.233487 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:02:14.233487 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:02:14.234691 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:02:14.234691 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:02:14.234691 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:02:14.234691 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:02:14.234691 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 12:02:14.234691 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 12:02:14.234691 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 12:02:14.234691 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 12:02:14.234691 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 12:02:14.234691 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 12:02:14.234691 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 12:02:14.234691 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 25 12:02:14.234691 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 349/697] 10.222.0.42 () {56 vars in 1096 bytes} [Mon Jul 25 12:02:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 12:02:14.240068 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14222a12-b681-4e89-8a7e-caf7f23aeeb4 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:14.240946 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8878b7-a276-4fb8-ba3a-ee5ed6361597 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:14.241853 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8878b7-a276-4fb8-ba3a-ee5ed6361597 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:14.242346 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8878b7-a276-4fb8-ba3a-ee5ed6361597 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:14.242883 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8878b7-a276-4fb8-ba3a-ee5ed6361597 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:14.244128 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8878b7-a276-4fb8-ba3a-ee5ed6361597 None admin] RBAC: Adding query filter params (role.id=79dec36aee5b4322bf652041c25ca6e4, scope.project.id=b9f62b8f37304b8a9a4bb54205347577, user.id=7bd782ea66364eeb96592096eae63d4f) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:02:14.244605 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8878b7-a276-4fb8-ba3a-ee5ed6361597 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:14.246307 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8878b7-a276-4fb8-ba3a-ee5ed6361597 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:14.253811 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 349/698] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Jul 25 12:02:14 2022] GET /identity/v3/role_assignments?user.id=7bd782ea66364eeb96592096eae63d4f&scope.project.id=b9f62b8f37304b8a9a4bb54205347577&role.id=79dec36aee5b4322bf652041c25ca6e4 => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 12:02:14.309865 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:14.368014 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14222a12-b681-4e89-8a7e-caf7f23aeeb4 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:14.417817 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1a6211-e8fa-4315-9882-3a8156ecd89a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:14.418876 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1a6211-e8fa-4315-9882-3a8156ecd89a None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:14.419397 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1a6211-e8fa-4315-9882-3a8156ecd89a None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:14.419887 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1a6211-e8fa-4315-9882-3a8156ecd89a None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:14.420888 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1a6211-e8fa-4315-9882-3a8156ecd89a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:02:14.421389 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1a6211-e8fa-4315-9882-3a8156ecd89a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:14.423017 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1a6211-e8fa-4315-9882-3a8156ecd89a None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:14.433069 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 350/699] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Jul 25 12:02:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 12:02:14.438141 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8878b7-a276-4fb8-ba3a-ee5ed6361597 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:14.492578 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:14.547073 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8878b7-a276-4fb8-ba3a-ee5ed6361597 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:14.583050 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306e8139-f7b6-4455-9de4-32402e731adb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:14.583814 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306e8139-f7b6-4455-9de4-32402e731adb None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:14.584105 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306e8139-f7b6-4455-9de4-32402e731adb None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:14.584452 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306e8139-f7b6-4455-9de4-32402e731adb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:14.596478 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306e8139-f7b6-4455-9de4-32402e731adb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:14.597941 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306e8139-f7b6-4455-9de4-32402e731adb None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:14.615729 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-306e8139-f7b6-4455-9de4-32402e731adb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:02:14.615729 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:02:14.615729 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:02:14.615729 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:02:14.615729 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:02:14.615729 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:02:14.615729 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:02:14.615729 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:02:14.615729 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:02:14.615729 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:02:14.615729 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:02:14.615729 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:02:14.615729 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 12:02:14.615729 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 12:02:14.615729 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 12:02:14.616653 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 12:02:14.616653 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:02:14.616653 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:02:14.616653 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:02:14.616653 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:02:14.616653 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:02:14.616653 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:02:14.616653 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:02:14.616653 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jul 25 12:02:14.616653 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:02:14.616653 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:02:14.616653 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:02:14.616653 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:02:14.616653 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:02:14.616653 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:02:14.616653 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:02:14.616653 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:02:14.617540 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 12:02:14.617540 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 12:02:14.617540 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:02:14.617540 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jul 25 12:02:14.617540 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 350/700] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jul 25 12:02:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 12:02:14.623814 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1a6211-e8fa-4315-9882-3a8156ecd89a None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:14.731695 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1a6211-e8fa-4315-9882-3a8156ecd89a None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:14.749654 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:14.779008 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576b3431-9e15-408c-9aa4-2b2d14742615 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:14.782622 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576b3431-9e15-408c-9aa4-2b2d14742615 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:14.782622 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576b3431-9e15-408c-9aa4-2b2d14742615 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:14.782622 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576b3431-9e15-408c-9aa4-2b2d14742615 None admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:14.782622 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576b3431-9e15-408c-9aa4-2b2d14742615 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:02:14.782622 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576b3431-9e15-408c-9aa4-2b2d14742615 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:14.784058 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576b3431-9e15-408c-9aa4-2b2d14742615 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:14.798102 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 351/701] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jul 25 12:02:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 12:02:14.803535 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306e8139-f7b6-4455-9de4-32402e731adb None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:14.896060 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a23b54-d1de-444c-8f94-6c52d49fb4cf None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:14.896364 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a23b54-d1de-444c-8f94-6c52d49fb4cf None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:14.896569 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a23b54-d1de-444c-8f94-6c52d49fb4cf None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:14.899670 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 352/702] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 25 12:02:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 12:02:14.907579 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb1a01b-5e2b-49a6-a6a8-bbc177d47e2c None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:14.907579 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb1a01b-5e2b-49a6-a6a8-bbc177d47e2c None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:14.907579 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb1a01b-5e2b-49a6-a6a8-bbc177d47e2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:14.917702 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306e8139-f7b6-4455-9de4-32402e731adb None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:14.919861 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:14.926859 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-bbb1a01b-5e2b-49a6-a6a8-bbc177d47e2c None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:02:14.929635 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-bbb1a01b-5e2b-49a6-a6a8-bbc177d47e2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:02:14.982730 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2941d4-0514-41da-89fb-3846b84aea57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:14.983942 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2941d4-0514-41da-89fb-3846b84aea57 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:14.984456 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2941d4-0514-41da-89fb-3846b84aea57 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:14.984957 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2941d4-0514-41da-89fb-3846b84aea57 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:15.007025 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2941d4-0514-41da-89fb-3846b84aea57 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:15.008905 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2941d4-0514-41da-89fb-3846b84aea57 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:15.031115 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-5d2941d4-0514-41da-89fb-3846b84aea57 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 25 12:02:15.031115 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:02:15.031115 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:02:15.031115 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:02:15.031115 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:02:15.031115 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:02:15.031115 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:02:15.031115 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:02:15.031115 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:02:15.031115 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:02:15.031115 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:02:15.031115 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:02:15.031115 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 12:02:15.031115 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 12:02:15.031115 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 12:02:15.031115 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 12:02:15.032186 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:02:15.032186 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:02:15.032186 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 12:02:15.032186 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 12:02:15.032186 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 12:02:15.032186 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 12:02:15.032186 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:02:15.032186 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:02:15.032186 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:02:15.032186 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:02:15.032186 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:02:15.032186 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jul 25 12:02:15.032186 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:02:15.032186 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:02:15.032186 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:02:15.032186 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:02:15.032186 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:02:15.032186 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:02:15.033817 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:02:15.033817 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:02:15.033817 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 12:02:15.033817 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 12:02:15.033817 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 12:02:15.033817 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 12:02:15.033817 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 12:02:15.033817 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 12:02:15.033817 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 25 12:02:15.033817 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jul 25 12:02:15.047367 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 351/703] 10.222.0.42 () {56 vars in 1093 bytes} [Mon Jul 25 12:02:14 2022] GET /identity/v3/users/nova => generated 82 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:02:15.053203 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2941d4-0514-41da-89fb-3846b84aea57 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:15.067591 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 353/704] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 25 12:02:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 25 12:02:15.102477 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb1a01b-5e2b-49a6-a6a8-bbc177d47e2c None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:15.202029 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2941d4-0514-41da-89fb-3846b84aea57 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:15.272644 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75405cda-b5de-4f86-815f-58433f9cd302 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:15.273422 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75405cda-b5de-4f86-815f-58433f9cd302 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:15.273716 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75405cda-b5de-4f86-815f-58433f9cd302 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:15.273997 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75405cda-b5de-4f86-815f-58433f9cd302 None admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:15.274841 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75405cda-b5de-4f86-815f-58433f9cd302 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:02:15.275159 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75405cda-b5de-4f86-815f-58433f9cd302 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:15.275266 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb1a01b-5e2b-49a6-a6a8-bbc177d47e2c None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:15.276782 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75405cda-b5de-4f86-815f-58433f9cd302 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:15.283848 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:15.314682 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 352/705] 10.222.0.42 () {56 vars in 1133 bytes} [Mon Jul 25 12:02:15 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 12:02:15.321829 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75405cda-b5de-4f86-815f-58433f9cd302 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:15.330245 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a37979b-cd37-4443-b576-fc2d14ffe7be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:15.331068 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a37979b-cd37-4443-b576-fc2d14ffe7be None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:15.331355 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a37979b-cd37-4443-b576-fc2d14ffe7be None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:15.331661 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a37979b-cd37-4443-b576-fc2d14ffe7be None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:15.336281 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a37979b-cd37-4443-b576-fc2d14ffe7be None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:15.337682 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a37979b-cd37-4443-b576-fc2d14ffe7be None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:15.367884 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-1a37979b-cd37-4443-b576-fc2d14ffe7be None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 25 12:02:15.367884 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:02:15.367884 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 12:02:15.367884 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 12:02:15.367884 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 12:02:15.367884 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 12:02:15.367884 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 12:02:15.367884 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 12:02:15.367884 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 25 12:02:15.367884 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 25 12:02:15.367884 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 12:02:15.367884 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 25 12:02:15.367884 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:02:15.367884 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:02:15.367884 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:02:15.367884 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:02:15.368945 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:02:15.368945 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:02:15.368945 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:02:15.368945 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:02:15.368945 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:02:15.368945 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:02:15.368945 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:02:15.368945 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 12:02:15.368945 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 12:02:15.368945 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 12:02:15.368945 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 12:02:15.368945 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:02:15.368945 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:02:15.368945 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:02:15.368945 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:02:15.368945 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:02:15.369810 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:02:15.369810 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:02:15.369810 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 25 12:02:15.369810 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:02:15.369810 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:02:15.369810 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:02:15.369810 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:02:15.369810 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:02:15.369810 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:02:15.369810 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:02:15.369810 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:02:15.369810 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 12:02:15.369810 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 12:02:15.369810 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 25 12:02:15.369810 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 25 12:02:15.370596 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:15.378514 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 354/706] 10.222.0.42 () {56 vars in 1129 bytes} [Mon Jul 25 12:02:15 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 12:02:15.382376 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a37979b-cd37-4443-b576-fc2d14ffe7be None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:15.538102 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75405cda-b5de-4f86-815f-58433f9cd302 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:15.551932 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a37979b-cd37-4443-b576-fc2d14ffe7be None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:15.596546 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cbeaf2-3691-45f7-a16b-cfc8ef45aa2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:15.597504 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cbeaf2-3691-45f7-a16b-cfc8ef45aa2e None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:15.597796 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cbeaf2-3691-45f7-a16b-cfc8ef45aa2e None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:15.598335 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cbeaf2-3691-45f7-a16b-cfc8ef45aa2e None admin] PATH_INFO: `/v3/services` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:15.599065 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cbeaf2-3691-45f7-a16b-cfc8ef45aa2e None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:02:15.599379 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cbeaf2-3691-45f7-a16b-cfc8ef45aa2e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:15.600797 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cbeaf2-3691-45f7-a16b-cfc8ef45aa2e None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:15.606604 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 355/707] 10.222.0.42 () {56 vars in 1124 bytes} [Mon Jul 25 12:02:15 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:02:15.614363 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cbeaf2-3691-45f7-a16b-cfc8ef45aa2e None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:15.633174 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d71754-1a66-4125-92f5-24df1081945b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:15.633825 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d71754-1a66-4125-92f5-24df1081945b None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:15.634122 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d71754-1a66-4125-92f5-24df1081945b None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:15.634397 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d71754-1a66-4125-92f5-24df1081945b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:15.685722 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:15.753526 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d71754-1a66-4125-92f5-24df1081945b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:15.754977 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d71754-1a66-4125-92f5-24df1081945b None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:15.803082 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:15.816337 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cbeaf2-3691-45f7-a16b-cfc8ef45aa2e None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:15.879075 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b43336-2545-4dd4-94f4-4326f6383c1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:15.879878 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b43336-2545-4dd4-94f4-4326f6383c1f None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:15.880162 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b43336-2545-4dd4-94f4-4326f6383c1f None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:15.880438 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b43336-2545-4dd4-94f4-4326f6383c1f None admin] PATH_INFO: `/v3/services` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:15.881222 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b43336-2545-4dd4-94f4-4326f6383c1f None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:02:15.881590 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b43336-2545-4dd4-94f4-4326f6383c1f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:15.883218 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b43336-2545-4dd4-94f4-4326f6383c1f None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:15.891434 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 356/708] 10.222.0.42 () {56 vars in 1124 bytes} [Mon Jul 25 12:02:15 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:02:15.959576 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 353/709] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Jul 25 12:02:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 639 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 25 12:02:15.966887 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b43336-2545-4dd4-94f4-4326f6383c1f None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:16.112386 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:16.116495 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b43336-2545-4dd4-94f4-4326f6383c1f None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:16.164546 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f958c04-4453-43f1-92d9-65540dc6343e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:16.165346 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f958c04-4453-43f1-92d9-65540dc6343e None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:16.165628 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f958c04-4453-43f1-92d9-65540dc6343e None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:16.166168 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f958c04-4453-43f1-92d9-65540dc6343e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:16.187962 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f958c04-4453-43f1-92d9-65540dc6343e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:16.189485 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f958c04-4453-43f1-92d9-65540dc6343e None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:16.214103 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-0f958c04-4453-43f1-92d9-65540dc6343e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:02:16.214103 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:02:16.214103 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:02:16.214103 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:02:16.214103 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:02:16.214103 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:02:16.214103 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:02:16.214103 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:02:16.214103 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:02:16.214103 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:02:16.214103 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:02:16.214103 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:02:16.214103 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 12:02:16.214103 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 12:02:16.214103 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 12:02:16.215091 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 12:02:16.215091 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:02:16.215091 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:02:16.215091 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:02:16.215091 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:02:16.215091 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:02:16.215091 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:02:16.215091 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:02:16.215091 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 25 12:02:16.215091 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:02:16.215091 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:02:16.215091 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:02:16.215091 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:02:16.215091 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:02:16.215091 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:02:16.215091 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:02:16.215091 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:02:16.216056 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 12:02:16.216056 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 12:02:16.216056 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:02:16.216056 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 25 12:02:16.216056 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 357/710] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jul 25 12:02:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:02:16.221675 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d71754-1a66-4125-92f5-24df1081945b None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:16.238786 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9447f92-f819-42c3-94f5-45fa8c3b61b2 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:16.239118 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9447f92-f819-42c3-94f5-45fa8c3b61b2 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:16.239336 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9447f92-f819-42c3-94f5-45fa8c3b61b2 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:16.242544 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 358/711] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 25 12:02:16 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:02:16.246174 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64991479-5ed5-48fa-901e-bb63ba2f3cc4 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:16.246460 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64991479-5ed5-48fa-901e-bb63ba2f3cc4 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:16.246736 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64991479-5ed5-48fa-901e-bb63ba2f3cc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:16.272040 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-64991479-5ed5-48fa-901e-bb63ba2f3cc4 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:02:16.273559 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-64991479-5ed5-48fa-901e-bb63ba2f3cc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:02:16.361903 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 359/712] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 25 12:02:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 12:02:16.362565 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d71754-1a66-4125-92f5-24df1081945b None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:16.388942 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64991479-5ed5-48fa-901e-bb63ba2f3cc4 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:16.394421 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:16.426691 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485e0615-4eb2-4a70-8e3c-8f5be7fe0bf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:16.427781 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485e0615-4eb2-4a70-8e3c-8f5be7fe0bf1 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:16.429220 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485e0615-4eb2-4a70-8e3c-8f5be7fe0bf1 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:16.429730 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485e0615-4eb2-4a70-8e3c-8f5be7fe0bf1 None admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:16.430736 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485e0615-4eb2-4a70-8e3c-8f5be7fe0bf1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:02:16.431575 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485e0615-4eb2-4a70-8e3c-8f5be7fe0bf1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:16.433928 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485e0615-4eb2-4a70-8e3c-8f5be7fe0bf1 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:16.458117 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 354/713] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jul 25 12:02:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:02:16.467589 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485e0615-4eb2-4a70-8e3c-8f5be7fe0bf1 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:16.538287 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:16.559180 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64991479-5ed5-48fa-901e-bb63ba2f3cc4 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:16.596199 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c490107-38bf-4933-adaa-110a5f11523d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:16.597124 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c490107-38bf-4933-adaa-110a5f11523d None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:16.597487 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c490107-38bf-4933-adaa-110a5f11523d None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:16.597839 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c490107-38bf-4933-adaa-110a5f11523d None admin] PATH_INFO: `/v3/services/network` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:16.599021 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c490107-38bf-4933-adaa-110a5f11523d None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:16.600960 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c490107-38bf-4933-adaa-110a5f11523d None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:16.610775 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485e0615-4eb2-4a70-8e3c-8f5be7fe0bf1 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:16.616817 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-3c490107-38bf-4933-adaa-110a5f11523d None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 25 12:02:16.616817 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:02:16.616817 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 12:02:16.616817 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 12:02:16.616817 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 12:02:16.616817 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 12:02:16.616817 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 12:02:16.616817 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 12:02:16.616817 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 25 12:02:16.616817 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 25 12:02:16.616817 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 12:02:16.616817 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 25 12:02:16.616817 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:02:16.616817 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:02:16.616817 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:02:16.616817 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:02:16.616817 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:02:16.617903 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:02:16.617903 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:02:16.617903 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:02:16.617903 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:02:16.617903 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:02:16.617903 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:02:16.617903 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 12:02:16.617903 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 12:02:16.617903 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 12:02:16.617903 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 12:02:16.617903 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:02:16.617903 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:02:16.617903 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:02:16.617903 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:02:16.617903 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:02:16.617903 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:02:16.618790 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:02:16.618790 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 25 12:02:16.618790 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:02:16.618790 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:02:16.618790 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:02:16.618790 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:02:16.618790 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:02:16.618790 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:02:16.618790 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:02:16.618790 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:02:16.618790 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 12:02:16.618790 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 12:02:16.618790 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 25 12:02:16.618790 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 25 12:02:16.618790 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 360/714] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Jul 25 12:02:16 2022] GET /identity/v3/services/network => generated 88 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:02:16.620828 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a8397b-8135-4f07-aa9b-bc1d8baf07e9 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:16.623584 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a8397b-8135-4f07-aa9b-bc1d8baf07e9 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:16.623584 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a8397b-8135-4f07-aa9b-bc1d8baf07e9 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:16.624045 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 361/715] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 25 12:02:16 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:02:16.626184 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a8397b-8135-4f07-aa9b-bc1d8baf07e9 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:16.653031 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c32145e-7a29-400a-9fcf-9877d1cef6fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:16.653981 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c32145e-7a29-400a-9fcf-9877d1cef6fc None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:16.654373 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c32145e-7a29-400a-9fcf-9877d1cef6fc None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:16.655296 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c32145e-7a29-400a-9fcf-9877d1cef6fc None admin] PATH_INFO: `/v3/projects/service` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:16.666299 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c32145e-7a29-400a-9fcf-9877d1cef6fc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:16.667968 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c32145e-7a29-400a-9fcf-9877d1cef6fc None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:16.687125 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-7c32145e-7a29-400a-9fcf-9877d1cef6fc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 12:02:16.687125 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:02:16.687125 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:02:16.687125 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:02:16.687125 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:02:16.687125 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:02:16.687125 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:02:16.687125 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:02:16.687125 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:02:16.687125 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:02:16.687125 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:02:16.687125 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:02:16.687125 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 12:02:16.687125 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 12:02:16.687125 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 12:02:16.688293 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 12:02:16.688293 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:02:16.688293 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:02:16.688293 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:02:16.688293 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:02:16.688293 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:02:16.688293 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:02:16.688293 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:02:16.688293 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jul 25 12:02:16.688293 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:02:16.688293 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:02:16.688293 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:02:16.688293 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:02:16.688293 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:02:16.688293 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:02:16.688293 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:02:16.688293 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:02:16.689524 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 12:02:16.689524 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 12:02:16.689524 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 12:02:16.689524 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 12:02:16.689524 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 12:02:16.689524 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 12:02:16.689524 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 12:02:16.689524 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jul 25 12:02:16.689524 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 355/716] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Jul 25 12:02:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:02:16.690469 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e650dc56-7e81-463d-b220-4e742c04d640 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:16.690469 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e650dc56-7e81-463d-b220-4e742c04d640 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:16.690469 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e650dc56-7e81-463d-b220-4e742c04d640 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:16.721785 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-e650dc56-7e81-463d-b220-4e742c04d640 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:02:16.723295 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-e650dc56-7e81-463d-b220-4e742c04d640 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:02:16.756231 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a8397b-8135-4f07-aa9b-bc1d8baf07e9 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:16.764781 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:16.797435 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ddac42-811a-4afb-b467-78629942cb3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:16.798336 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ddac42-811a-4afb-b467-78629942cb3e None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:16.798871 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ddac42-811a-4afb-b467-78629942cb3e None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:16.799396 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ddac42-811a-4afb-b467-78629942cb3e None admin] PATH_INFO: `/v3/services` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:16.800417 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ddac42-811a-4afb-b467-78629942cb3e None admin] RBAC: Adding query filter params (name=network) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:02:16.800912 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ddac42-811a-4afb-b467-78629942cb3e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:16.802531 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ddac42-811a-4afb-b467-78629942cb3e None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:16.808118 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 362/717] 10.222.0.42 () {56 vars in 1112 bytes} [Mon Jul 25 12:02:16 2022] GET /identity/v3/services?name=network => generated 124 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 12:02:16.809138 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ddac42-811a-4afb-b467-78629942cb3e None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:16.816313 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 356/718] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 25 12:02:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 12:02:16.816313 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e650dc56-7e81-463d-b220-4e742c04d640 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:16.837179 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:16.934675 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e650dc56-7e81-463d-b220-4e742c04d640 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:16.969948 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ddac42-811a-4afb-b467-78629942cb3e None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:16.971379 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3c697d-9dae-4469-8791-63c5381f6a6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:16.972132 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3c697d-9dae-4469-8791-63c5381f6a6e None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:16.972359 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3c697d-9dae-4469-8791-63c5381f6a6e None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:16.972733 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3c697d-9dae-4469-8791-63c5381f6a6e None admin] PATH_INFO: `/v3/services` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:16.973499 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3c697d-9dae-4469-8791-63c5381f6a6e None admin] RBAC: Adding query filter params (type=network) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:02:16.973843 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3c697d-9dae-4469-8791-63c5381f6a6e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:16.975189 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3c697d-9dae-4469-8791-63c5381f6a6e None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:16.987684 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 357/719] 10.222.0.42 () {56 vars in 1112 bytes} [Mon Jul 25 12:02:16 2022] GET /identity/v3/services?type=network => generated 124 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:02:16.988765 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3c697d-9dae-4469-8791-63c5381f6a6e None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:17.018702 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90795a2e-5469-4a15-8ab8-c345973cf21c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:17.019801 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90795a2e-5469-4a15-8ab8-c345973cf21c None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:17.021521 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90795a2e-5469-4a15-8ab8-c345973cf21c None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:17.022052 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90795a2e-5469-4a15-8ab8-c345973cf21c None admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:17.024343 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90795a2e-5469-4a15-8ab8-c345973cf21c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:02:17.027153 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90795a2e-5469-4a15-8ab8-c345973cf21c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:17.028952 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90795a2e-5469-4a15-8ab8-c345973cf21c None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:17.046905 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 363/720] 10.222.0.42 () {56 vars in 1148 bytes} [Mon Jul 25 12:02:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:02:17.048741 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f981e7-0390-40c7-8c37-1b6cd37429f9 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:17.049292 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f981e7-0390-40c7-8c37-1b6cd37429f9 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:17.049776 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f981e7-0390-40c7-8c37-1b6cd37429f9 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:17.051715 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 364/721] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 25 12:02:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:02:17.053317 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f981e7-0390-40c7-8c37-1b6cd37429f9 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:17.064760 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:17.159018 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:17.159972 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3c697d-9dae-4469-8791-63c5381f6a6e None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:17.197199 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ef3131-0965-4721-b50d-96946e542625 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:17.197920 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ef3131-0965-4721-b50d-96946e542625 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:17.198207 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ef3131-0965-4721-b50d-96946e542625 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:17.198487 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ef3131-0965-4721-b50d-96946e542625 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:17.204002 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f981e7-0390-40c7-8c37-1b6cd37429f9 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:17.209753 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ef3131-0965-4721-b50d-96946e542625 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 12:02:17.217724 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-00ef3131-0965-4721-b50d-96946e542625 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 25 12:02:17.217724 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:02:17.217724 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:02:17.217724 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:02:17.217724 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:02:17.217724 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:02:17.217724 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:02:17.217724 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:02:17.217724 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:02:17.217724 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:02:17.217724 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:02:17.217724 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:02:17.217724 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 12:02:17.217724 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 12:02:17.217724 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 12:02:17.217724 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 12:02:17.218746 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 12:02:17.218746 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 12:02:17.218746 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 12:02:17.218746 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 12:02:17.218746 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:02:17.218746 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:02:17.218746 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:02:17.218746 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:02:17.218746 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:02:17.218746 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:02:17.218746 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:02:17.218746 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jul 25 12:02:17.218746 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:02:17.218746 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:02:17.218746 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:02:17.218746 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:02:17.219658 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:02:17.219658 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:02:17.219658 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:02:17.219658 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:02:17.219658 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 12:02:17.219658 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 12:02:17.219658 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 12:02:17.219658 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 12:02:17.219658 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 12:02:17.219658 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 12:02:17.219658 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 25 12:02:17.219658 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jul 25 12:02:17.219658 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 358/722] 10.222.0.42 () {56 vars in 1102 bytes} [Mon Jul 25 12:02:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:02:17.220950 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64f4a10-c000-4991-8233-caa59b817072 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:17.220950 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64f4a10-c000-4991-8233-caa59b817072 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:17.221243 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64f4a10-c000-4991-8233-caa59b817072 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:17.240164 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-a64f4a10-c000-4991-8233-caa59b817072 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:02:17.241359 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-a64f4a10-c000-4991-8233-caa59b817072 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:02:17.250886 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cc374c-e22f-4be6-93ca-9ed98916f9c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:17.252816 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cc374c-e22f-4be6-93ca-9ed98916f9c5 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:17.253734 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cc374c-e22f-4be6-93ca-9ed98916f9c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:17.254225 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cc374c-e22f-4be6-93ca-9ed98916f9c5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:17.255570 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69cc374c-e22f-4be6-93ca-9ed98916f9c5 None admin] RBAC: Adding query filter params (role.id=a7d20885b6d94928a34b3cc5b3a00006, scope.project.id=b9f62b8f37304b8a9a4bb54205347577, user.id=f67142a1193d47c7bd33d7040268233c) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:02:17.256159 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69cc374c-e22f-4be6-93ca-9ed98916f9c5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:17.258591 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69cc374c-e22f-4be6-93ca-9ed98916f9c5 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:17.271754 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 365/723] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Jul 25 12:02:17 2022] GET /identity/v3/role_assignments?user.id=f67142a1193d47c7bd33d7040268233c&scope.project.id=b9f62b8f37304b8a9a4bb54205347577&role.id=a7d20885b6d94928a34b3cc5b3a00006 => generated 259 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:02:17.271941 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cc374c-e22f-4be6-93ca-9ed98916f9c5 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:17.368989 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 359/724] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 25 12:02:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 12:02:17.414467 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64f4a10-c000-4991-8233-caa59b817072 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:17.454353 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cc374c-e22f-4be6-93ca-9ed98916f9c5 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:17.484846 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:17.492474 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:17.496231 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef12c189-403a-4c6f-b624-afbcd23957c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:17.497008 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef12c189-403a-4c6f-b624-afbcd23957c8 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:17.497301 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef12c189-403a-4c6f-b624-afbcd23957c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:17.497612 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef12c189-403a-4c6f-b624-afbcd23957c8 None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:17.498418 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef12c189-403a-4c6f-b624-afbcd23957c8 None admin] RBAC: Adding query filter params (name=service) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:02:17.498732 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef12c189-403a-4c6f-b624-afbcd23957c8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:17.500223 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef12c189-403a-4c6f-b624-afbcd23957c8 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:17.510542 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 366/725] 10.222.0.42 () {56 vars in 1103 bytes} [Mon Jul 25 12:02:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:02:17.516460 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef12c189-403a-4c6f-b624-afbcd23957c8 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:17.659640 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64f4a10-c000-4991-8233-caa59b817072 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:17.711367 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef12c189-403a-4c6f-b624-afbcd23957c8 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:17.729173 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe109a7-2f46-4d44-b068-3e9860b2195c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:17.729978 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe109a7-2f46-4d44-b068-3e9860b2195c None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:17.730260 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe109a7-2f46-4d44-b068-3e9860b2195c None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:17.730533 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe109a7-2f46-4d44-b068-3e9860b2195c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:17.763568 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe109a7-2f46-4d44-b068-3e9860b2195c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 12:02:17.778121 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-abe109a7-2f46-4d44-b068-3e9860b2195c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 12:02:17.778121 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:02:17.778121 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:02:17.778121 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:02:17.778121 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:02:17.778121 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:02:17.778121 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:02:17.778121 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:02:17.778121 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:02:17.778121 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:02:17.778121 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:02:17.778121 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:02:17.778121 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 12:02:17.778121 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 12:02:17.778121 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 12:02:17.778121 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 12:02:17.779199 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 12:02:17.779199 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 12:02:17.779199 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 12:02:17.779199 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 12:02:17.779199 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:02:17.779199 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:02:17.779199 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:02:17.779199 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:02:17.779199 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:02:17.779199 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:02:17.779199 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:02:17.779199 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jul 25 12:02:17.779199 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:02:17.779199 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:02:17.779199 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:02:17.779199 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:02:17.780113 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:02:17.780113 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:02:17.780113 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:02:17.780113 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:02:17.780113 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 12:02:17.780113 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 12:02:17.780113 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 12:02:17.780113 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 12:02:17.780113 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 12:02:17.780113 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 12:02:17.780113 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 12:02:17.780113 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jul 25 12:02:17.788160 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 360/726] 10.222.0.42 () {56 vars in 1096 bytes} [Mon Jul 25 12:02:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 372 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:02:17.788160 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe109a7-2f46-4d44-b068-3e9860b2195c None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:17.790327 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fd0c3c-ee41-4803-ba69-a158e9b187ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:17.791384 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fd0c3c-ee41-4803-ba69-a158e9b187ef None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:17.796062 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fd0c3c-ee41-4803-ba69-a158e9b187ef None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:17.796358 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fd0c3c-ee41-4803-ba69-a158e9b187ef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:17.821179 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fd0c3c-ee41-4803-ba69-a158e9b187ef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:17.825773 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fd0c3c-ee41-4803-ba69-a158e9b187ef None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:17.866762 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-a8fd0c3c-ee41-4803-ba69-a158e9b187ef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:02:17.866762 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:02:17.866762 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:02:17.866762 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:02:17.866762 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:02:17.866762 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:02:17.866762 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:02:17.866762 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:02:17.866762 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:02:17.866762 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:02:17.866762 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:02:17.866762 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:02:17.866762 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 12:02:17.866762 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 12:02:17.866762 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 12:02:17.867760 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 12:02:17.867760 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:02:17.867760 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:02:17.867760 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:02:17.867760 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:02:17.867760 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:02:17.867760 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:02:17.867760 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:02:17.867760 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 25 12:02:17.867760 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:02:17.867760 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:02:17.867760 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:02:17.867760 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:02:17.867760 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:02:17.867760 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:02:17.867760 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:02:17.867760 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:02:17.868668 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 12:02:17.868668 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 12:02:17.868668 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:02:17.868668 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 25 12:02:17.878168 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 367/727] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jul 25 12:02:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 358 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:02:17.879973 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fd0c3c-ee41-4803-ba69-a158e9b187ef None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:17.995953 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:17.996124 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:18.048016 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe109a7-2f46-4d44-b068-3e9860b2195c None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:18.120936 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fd0c3c-ee41-4803-ba69-a158e9b187ef None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:18.136575 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16807df-c5b8-4100-9b8a-dd84a0af178a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:18.137261 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16807df-c5b8-4100-9b8a-dd84a0af178a None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:18.137547 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16807df-c5b8-4100-9b8a-dd84a0af178a None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:18.137827 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16807df-c5b8-4100-9b8a-dd84a0af178a None admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:18.138684 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16807df-c5b8-4100-9b8a-dd84a0af178a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:02:18.138993 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16807df-c5b8-4100-9b8a-dd84a0af178a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:18.140390 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16807df-c5b8-4100-9b8a-dd84a0af178a None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:18.164994 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 361/728] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Jul 25 12:02:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 378 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:02:18.172901 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16807df-c5b8-4100-9b8a-dd84a0af178a None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:18.206496 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f99c89-5931-4391-b565-dd8f4c18ed31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:18.207606 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f99c89-5931-4391-b565-dd8f4c18ed31 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:18.207888 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f99c89-5931-4391-b565-dd8f4c18ed31 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:18.208164 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f99c89-5931-4391-b565-dd8f4c18ed31 None admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:18.210180 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f99c89-5931-4391-b565-dd8f4c18ed31 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:02:18.210180 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f99c89-5931-4391-b565-dd8f4c18ed31 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:18.210577 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f99c89-5931-4391-b565-dd8f4c18ed31 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:18.241323 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 368/729] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jul 25 12:02:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 359 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:02:18.247262 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f99c89-5931-4391-b565-dd8f4c18ed31 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:18.353688 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16807df-c5b8-4100-9b8a-dd84a0af178a None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:18.402189 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f99c89-5931-4391-b565-dd8f4c18ed31 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:18.410686 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:18.417149 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32bc5a9-d5da-462e-ab79-0709c1b9e6f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:18.418021 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32bc5a9-d5da-462e-ab79-0709c1b9e6f6 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:18.418308 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32bc5a9-d5da-462e-ab79-0709c1b9e6f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:18.418643 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32bc5a9-d5da-462e-ab79-0709c1b9e6f6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:18.434882 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32bc5a9-d5da-462e-ab79-0709c1b9e6f6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:18.436354 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32bc5a9-d5da-462e-ab79-0709c1b9e6f6 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:18.439310 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad2cc05-a4a3-4338-80fa-d9fd98b5c312 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:18.440156 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad2cc05-a4a3-4338-80fa-d9fd98b5c312 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:18.440438 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad2cc05-a4a3-4338-80fa-d9fd98b5c312 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:18.440715 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad2cc05-a4a3-4338-80fa-d9fd98b5c312 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:18.447997 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad2cc05-a4a3-4338-80fa-d9fd98b5c312 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:18.449445 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad2cc05-a4a3-4338-80fa-d9fd98b5c312 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:18.455899 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:18.460879 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-e32bc5a9-d5da-462e-ab79-0709c1b9e6f6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:02:18.460879 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:02:18.460879 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:02:18.460879 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:02:18.460879 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:02:18.460879 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:02:18.460879 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:02:18.460879 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:02:18.460879 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:02:18.460879 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:02:18.460879 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:02:18.460879 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:02:18.460879 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 12:02:18.460879 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 12:02:18.460879 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 12:02:18.462919 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 12:02:18.462919 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:02:18.462919 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:02:18.462919 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:02:18.462919 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:02:18.462919 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:02:18.462919 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:02:18.462919 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:02:18.462919 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jul 25 12:02:18.462919 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:02:18.462919 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:02:18.462919 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:02:18.462919 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:02:18.462919 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:02:18.462919 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:02:18.462919 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:02:18.462919 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:02:18.463839 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 12:02:18.463839 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 12:02:18.463839 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:02:18.463839 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jul 25 12:02:18.468849 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 362/730] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jul 25 12:02:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 12:02:18.470832 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08f7e36-d2dc-4909-b54c-7f71fe5bb76e None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:18.471224 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08f7e36-d2dc-4909-b54c-7f71fe5bb76e None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:18.471648 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08f7e36-d2dc-4909-b54c-7f71fe5bb76e None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:18.474545 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-fad2cc05-a4a3-4338-80fa-d9fd98b5c312 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 25 12:02:18.474545 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:02:18.474545 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:02:18.474545 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:02:18.474545 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:02:18.474545 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:02:18.474545 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:02:18.474545 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:02:18.474545 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:02:18.474545 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:02:18.474545 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:02:18.474545 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:02:18.474545 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 12:02:18.474545 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 12:02:18.474545 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 12:02:18.474545 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 12:02:18.475439 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:02:18.475439 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:02:18.475439 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 12:02:18.475439 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 12:02:18.475439 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 12:02:18.475439 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 12:02:18.475439 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:02:18.475439 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:02:18.475439 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:02:18.475439 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:02:18.475439 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:02:18.475439 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 25 12:02:18.475439 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:02:18.475439 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:02:18.475439 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:02:18.475439 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:02:18.475439 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:02:18.475439 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:02:18.476391 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:02:18.476391 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:02:18.476391 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 12:02:18.476391 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 12:02:18.476391 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 12:02:18.476391 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 12:02:18.476391 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 12:02:18.476391 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 12:02:18.476391 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 25 12:02:18.476391 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 25 12:02:18.477173 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 363/731] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 25 12:02:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:02:18.477173 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08f7e36-d2dc-4909-b54c-7f71fe5bb76e None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:18.478816 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 369/732] 10.222.0.42 () {56 vars in 1117 bytes} [Mon Jul 25 12:02:18 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:02:18.480208 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332ea708-f9d6-492c-b90a-5f22200b5951 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:18.480704 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332ea708-f9d6-492c-b90a-5f22200b5951 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:18.481199 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332ea708-f9d6-492c-b90a-5f22200b5951 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:18.500840 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-332ea708-f9d6-492c-b90a-5f22200b5951 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:02:18.502328 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-332ea708-f9d6-492c-b90a-5f22200b5951 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:02:18.570184 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 370/733] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 25 12:02:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 12:02:18.571398 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332ea708-f9d6-492c-b90a-5f22200b5951 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:18.604305 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08f7e36-d2dc-4909-b54c-7f71fe5bb76e None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:18.656848 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97f1a17-7dae-4ccb-bfd7-b36d6a7f391f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:18.658296 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97f1a17-7dae-4ccb-bfd7-b36d6a7f391f None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:18.658579 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97f1a17-7dae-4ccb-bfd7-b36d6a7f391f None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:18.658858 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97f1a17-7dae-4ccb-bfd7-b36d6a7f391f None admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:18.659598 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97f1a17-7dae-4ccb-bfd7-b36d6a7f391f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:02:18.659911 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97f1a17-7dae-4ccb-bfd7-b36d6a7f391f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:18.661288 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97f1a17-7dae-4ccb-bfd7-b36d6a7f391f None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:18.679053 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 364/734] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jul 25 12:02:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:02:18.680331 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97f1a17-7dae-4ccb-bfd7-b36d6a7f391f None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:18.693764 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:18.699990 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332ea708-f9d6-492c-b90a-5f22200b5951 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:18.740305 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81a4c36-cdfa-4cca-b08c-58ced65dc6f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:18.740986 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81a4c36-cdfa-4cca-b08c-58ced65dc6f0 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:18.741269 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81a4c36-cdfa-4cca-b08c-58ced65dc6f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:18.741545 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81a4c36-cdfa-4cca-b08c-58ced65dc6f0 None admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:18.742392 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81a4c36-cdfa-4cca-b08c-58ced65dc6f0 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:02:18.742698 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81a4c36-cdfa-4cca-b08c-58ced65dc6f0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:18.744258 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81a4c36-cdfa-4cca-b08c-58ced65dc6f0 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:18.764747 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:18.769368 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 371/735] 10.222.0.42 () {56 vars in 1149 bytes} [Mon Jul 25 12:02:18 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:02:18.771578 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81a4c36-cdfa-4cca-b08c-58ced65dc6f0 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:18.815821 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97f1a17-7dae-4ccb-bfd7-b36d6a7f391f None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:18.853838 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf55430-c5e2-4e43-b10f-0b407b18c34d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:18.854510 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf55430-c5e2-4e43-b10f-0b407b18c34d None admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:18.854881 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf55430-c5e2-4e43-b10f-0b407b18c34d None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:18.855073 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf55430-c5e2-4e43-b10f-0b407b18c34d None admin] PATH_INFO: `/v3/services` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:18.855845 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf55430-c5e2-4e43-b10f-0b407b18c34d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:18.857213 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf55430-c5e2-4e43-b10f-0b407b18c34d None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:18.883069 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 365/736] 10.222.0.42 () {60 vars in 1141 bytes} [Mon Jul 25 12:02:18 2022] POST /identity/v3/services => generated 251 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:02:18.883947 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf55430-c5e2-4e43-b10f-0b407b18c34d None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:18.918622 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81a4c36-cdfa-4cca-b08c-58ced65dc6f0 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:18.965966 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e8a313-f293-4cc1-a8f8-09ea0ca83ce4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:18.967205 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e8a313-f293-4cc1-a8f8-09ea0ca83ce4 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:18.967205 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e8a313-f293-4cc1-a8f8-09ea0ca83ce4 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:18.967720 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e8a313-f293-4cc1-a8f8-09ea0ca83ce4 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:18.976718 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e8a313-f293-4cc1-a8f8-09ea0ca83ce4 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:18.978733 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e8a313-f293-4cc1-a8f8-09ea0ca83ce4 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:18.994511 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-e2e8a313-f293-4cc1-a8f8-09ea0ca83ce4 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 25 12:02:18.994511 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:02:18.994511 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:02:18.994511 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:02:18.994511 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:02:18.994511 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:02:18.994511 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:02:18.994511 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:02:18.994511 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:02:18.994511 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:02:18.994511 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:02:18.994511 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:02:18.994511 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 12:02:18.994511 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 12:02:18.994511 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 12:02:18.994511 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 12:02:18.995740 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:02:18.995740 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:02:18.995740 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 12:02:18.995740 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 12:02:18.995740 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 12:02:18.995740 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 12:02:18.995740 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:02:18.995740 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:02:18.995740 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:02:18.995740 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:02:18.995740 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:02:18.995740 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 25 12:02:18.995740 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:02:18.995740 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:02:18.995740 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:02:18.995740 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:02:18.995740 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:02:18.995740 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:02:18.996903 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:02:18.996903 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:02:18.996903 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 12:02:18.996903 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 12:02:18.996903 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 12:02:18.996903 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 12:02:18.996903 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 12:02:18.996903 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 12:02:18.996903 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 25 12:02:18.996903 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 25 12:02:19.003201 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 372/737] 10.222.0.42 () {56 vars in 1096 bytes} [Mon Jul 25 12:02:18 2022] GET /identity/v3/users/swift => generated 83 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:02:19.004204 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e8a313-f293-4cc1-a8f8-09ea0ca83ce4 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:19.018804 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:19.023338 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf55430-c5e2-4e43-b10f-0b407b18c34d None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:19.049852 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:19.071779 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c13fcf-88f8-40c8-b6e5-209f83659d08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:19.073537 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c13fcf-88f8-40c8-b6e5-209f83659d08 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:19.074016 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c13fcf-88f8-40c8-b6e5-209f83659d08 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:19.074480 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c13fcf-88f8-40c8-b6e5-209f83659d08 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:19.161784 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e8a313-f293-4cc1-a8f8-09ea0ca83ce4 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:19.166249 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c13fcf-88f8-40c8-b6e5-209f83659d08 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:19.168019 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c13fcf-88f8-40c8-b6e5-209f83659d08 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:19.201514 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f784b83-9455-423e-b428-7eddaa36bcce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:19.202195 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f784b83-9455-423e-b428-7eddaa36bcce None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:19.202490 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f784b83-9455-423e-b428-7eddaa36bcce None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:19.202742 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f784b83-9455-423e-b428-7eddaa36bcce None admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:19.203625 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f784b83-9455-423e-b428-7eddaa36bcce None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:02:19.203942 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f784b83-9455-423e-b428-7eddaa36bcce None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:19.205939 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f784b83-9455-423e-b428-7eddaa36bcce None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:19.229196 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 373/738] 10.222.0.42 () {56 vars in 1135 bytes} [Mon Jul 25 12:02:19 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 12:02:19.230338 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f784b83-9455-423e-b428-7eddaa36bcce None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:19.304066 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:19.305277 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 366/739] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Jul 25 12:02:18 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 12:02:19.312157 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c13fcf-88f8-40c8-b6e5-209f83659d08 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:19.337503 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:19.347585 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f784b83-9455-423e-b428-7eddaa36bcce None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:19.393431 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ef3ec9-f50c-4332-8a97-1716d7afd250 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:19.394581 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ef3ec9-f50c-4332-8a97-1716d7afd250 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:19.394881 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ef3ec9-f50c-4332-8a97-1716d7afd250 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:19.395169 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ef3ec9-f50c-4332-8a97-1716d7afd250 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:19.430192 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c13fcf-88f8-40c8-b6e5-209f83659d08 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:19.469355 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e5d038-0e50-4864-9feb-2f6d37e4e6fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:19.470185 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e5d038-0e50-4864-9feb-2f6d37e4e6fb None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:19.470476 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e5d038-0e50-4864-9feb-2f6d37e4e6fb None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:19.470757 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e5d038-0e50-4864-9feb-2f6d37e4e6fb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:19.486027 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e5d038-0e50-4864-9feb-2f6d37e4e6fb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:19.487627 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e5d038-0e50-4864-9feb-2f6d37e4e6fb None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:19.495358 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ef3ec9-f50c-4332-8a97-1716d7afd250 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:19.497859 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ef3ec9-f50c-4332-8a97-1716d7afd250 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:19.506072 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-b1e5d038-0e50-4864-9feb-2f6d37e4e6fb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:02:19.506072 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:02:19.506072 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:02:19.506072 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:02:19.506072 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:02:19.506072 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:02:19.506072 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:02:19.506072 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:02:19.506072 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:02:19.506072 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:02:19.506072 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:02:19.506072 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:02:19.506072 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 12:02:19.506072 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 12:02:19.506072 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 12:02:19.506984 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 12:02:19.506984 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:02:19.506984 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:02:19.506984 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:02:19.506984 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:02:19.506984 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:02:19.506984 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:02:19.506984 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:02:19.506984 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jul 25 12:02:19.506984 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:02:19.506984 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:02:19.506984 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:02:19.506984 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:02:19.506984 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:02:19.506984 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:02:19.506984 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:02:19.506984 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:02:19.507889 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 12:02:19.507889 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 12:02:19.507889 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:02:19.507889 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jul 25 12:02:19.507889 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 367/740] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jul 25 12:02:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:02:19.508958 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a268e0da-19e3-4075-95f1-f65a484f9ac6 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:19.509711 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a268e0da-19e3-4075-95f1-f65a484f9ac6 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:19.509711 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a268e0da-19e3-4075-95f1-f65a484f9ac6 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:19.515118 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 368/741] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 25 12:02:19 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:02:19.515985 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a268e0da-19e3-4075-95f1-f65a484f9ac6 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:19.593008 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:19.595697 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:19.635140 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 374/742] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Jul 25 12:02:19 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 12:02:19.636630 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e839159-963d-478a-8b0f-d0153326c840 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:19.636963 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e839159-963d-478a-8b0f-d0153326c840 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:19.637258 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e839159-963d-478a-8b0f-d0153326c840 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:19.638002 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a268e0da-19e3-4075-95f1-f65a484f9ac6 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:19.656862 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-0e839159-963d-478a-8b0f-d0153326c840 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:02:19.658201 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-0e839159-963d-478a-8b0f-d0153326c840 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:02:19.680256 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b627f04-84d4-4ec7-9d9f-4cca96af2fea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:19.680927 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b627f04-84d4-4ec7-9d9f-4cca96af2fea None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:19.681209 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b627f04-84d4-4ec7-9d9f-4cca96af2fea None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:19.681489 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b627f04-84d4-4ec7-9d9f-4cca96af2fea None admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:19.682225 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b627f04-84d4-4ec7-9d9f-4cca96af2fea None admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:02:19.682537 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b627f04-84d4-4ec7-9d9f-4cca96af2fea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:19.683965 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b627f04-84d4-4ec7-9d9f-4cca96af2fea None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:19.696293 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 369/743] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jul 25 12:02:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:02:19.697097 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b627f04-84d4-4ec7-9d9f-4cca96af2fea None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:19.739726 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 375/744] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 25 12:02:19 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 12:02:19.741372 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81955453-8680-4783-a16e-235534c2536a None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:19.741876 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81955453-8680-4783-a16e-235534c2536a None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:19.742355 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81955453-8680-4783-a16e-235534c2536a None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:19.743920 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 376/745] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 25 12:02:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:02:19.744631 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81955453-8680-4783-a16e-235534c2536a None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:19.821719 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b627f04-84d4-4ec7-9d9f-4cca96af2fea None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:19.852341 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:19.859288 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5e9581-e21e-4921-82ae-f1bcaf70c541 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:19.859451 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:19.860415 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5e9581-e21e-4921-82ae-f1bcaf70c541 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:19.860712 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5e9581-e21e-4921-82ae-f1bcaf70c541 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:19.861511 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5e9581-e21e-4921-82ae-f1bcaf70c541 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:19.866122 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81955453-8680-4783-a16e-235534c2536a None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:19.873084 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5e9581-e21e-4921-82ae-f1bcaf70c541 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:19.874531 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5e9581-e21e-4921-82ae-f1bcaf70c541 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:19.892403 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-fe5e9581-e21e-4921-82ae-f1bcaf70c541 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:02:19.892403 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:02:19.892403 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:02:19.892403 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:02:19.892403 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:02:19.892403 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:02:19.892403 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:02:19.892403 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:02:19.892403 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:02:19.892403 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:02:19.892403 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:02:19.892403 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:02:19.892403 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 12:02:19.892403 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 12:02:19.892403 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 12:02:19.893326 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 12:02:19.893326 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:02:19.893326 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:02:19.893326 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:02:19.893326 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:02:19.893326 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:02:19.893326 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:02:19.893326 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:02:19.893326 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jul 25 12:02:19.893326 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:02:19.893326 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:02:19.893326 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:02:19.893326 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:02:19.893326 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:02:19.893326 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:02:19.893326 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:02:19.893326 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:02:19.894227 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 12:02:19.894227 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 12:02:19.894227 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:02:19.894227 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jul 25 12:02:19.894227 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 370/746] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jul 25 12:02:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:02:19.894781 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b9d8ea-e7ca-4072-ab11-75f0cbbc7b81 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:19.895042 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b9d8ea-e7ca-4072-ab11-75f0cbbc7b81 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:19.895324 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b9d8ea-e7ca-4072-ab11-75f0cbbc7b81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:19.910662 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab419f80-0cd0-42b3-ace8-20ecd252d8fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:19.911788 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab419f80-0cd0-42b3-ace8-20ecd252d8fe None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:19.912289 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab419f80-0cd0-42b3-ace8-20ecd252d8fe None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:19.912749 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab419f80-0cd0-42b3-ace8-20ecd252d8fe None admin] PATH_INFO: `/v3/projects/service` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:19.913921 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-42b9d8ea-e7ca-4072-ab11-75f0cbbc7b81 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:02:19.915116 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-42b9d8ea-e7ca-4072-ab11-75f0cbbc7b81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:02:19.921017 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab419f80-0cd0-42b3-ace8-20ecd252d8fe None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:19.923120 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab419f80-0cd0-42b3-ace8-20ecd252d8fe None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:19.939063 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-ab419f80-0cd0-42b3-ace8-20ecd252d8fe None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 12:02:19.939063 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:02:19.939063 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:02:19.939063 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:02:19.939063 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:02:19.939063 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:02:19.939063 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:02:19.939063 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:02:19.939063 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:02:19.939063 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:02:19.939063 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:02:19.939063 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:02:19.939063 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 12:02:19.939063 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 12:02:19.939063 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 12:02:19.940207 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 12:02:19.940207 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:02:19.940207 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:02:19.940207 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:02:19.940207 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:02:19.940207 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:02:19.940207 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:02:19.940207 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:02:19.940207 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 25 12:02:19.940207 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:02:19.940207 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:02:19.940207 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:02:19.940207 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:02:19.940207 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:02:19.940207 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:02:19.940207 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:02:19.940207 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:02:19.941303 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 12:02:19.941303 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 12:02:19.941303 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 12:02:19.941303 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 12:02:19.941303 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 12:02:19.941303 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 12:02:19.941303 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 12:02:19.941303 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 25 12:02:19.943400 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 377/747] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Jul 25 12:02:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:02:19.944324 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab419f80-0cd0-42b3-ace8-20ecd252d8fe None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:19.991736 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 371/748] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 25 12:02:19 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 12:02:19.992656 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b9d8ea-e7ca-4072-ab11-75f0cbbc7b81 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:20.056157 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab419f80-0cd0-42b3-ace8-20ecd252d8fe None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:20.092781 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80937229-a623-4023-8583-ae0e578f7963 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:20.093448 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80937229-a623-4023-8583-ae0e578f7963 None admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:20.093737 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80937229-a623-4023-8583-ae0e578f7963 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:20.094018 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80937229-a623-4023-8583-ae0e578f7963 None admin] PATH_INFO: `/v3/services` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:20.094865 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80937229-a623-4023-8583-ae0e578f7963 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:20.096920 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80937229-a623-4023-8583-ae0e578f7963 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:20.111832 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b9d8ea-e7ca-4072-ab11-75f0cbbc7b81 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:20.123904 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 378/749] 10.222.0.42 () {60 vars in 1141 bytes} [Mon Jul 25 12:02:19 2022] POST /identity/v3/services => generated 240 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:02:20.123904 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80937229-a623-4023-8583-ae0e578f7963 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:20.141998 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:20.144831 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:20.191501 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866a2b91-61cb-48ef-8f9c-200821f2661e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:20.192246 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866a2b91-61cb-48ef-8f9c-200821f2661e None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:20.192536 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866a2b91-61cb-48ef-8f9c-200821f2661e None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:20.192820 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866a2b91-61cb-48ef-8f9c-200821f2661e None admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:20.193706 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866a2b91-61cb-48ef-8f9c-200821f2661e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:02:20.194046 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866a2b91-61cb-48ef-8f9c-200821f2661e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:20.195623 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866a2b91-61cb-48ef-8f9c-200821f2661e None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:20.212731 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 372/750] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jul 25 12:02:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:02:20.213693 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866a2b91-61cb-48ef-8f9c-200821f2661e None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:20.253112 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80937229-a623-4023-8583-ae0e578f7963 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:20.294454 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfef610-7013-4be5-838f-cd046ff18580 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:20.295598 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfef610-7013-4be5-838f-cd046ff18580 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:20.296135 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfef610-7013-4be5-838f-cd046ff18580 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:20.296621 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfef610-7013-4be5-838f-cd046ff18580 None admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:20.299752 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbfef610-7013-4be5-838f-cd046ff18580 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:02:20.300078 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbfef610-7013-4be5-838f-cd046ff18580 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:20.301532 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbfef610-7013-4be5-838f-cd046ff18580 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:20.316567 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 379/751] 10.222.0.42 () {56 vars in 1148 bytes} [Mon Jul 25 12:02:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:02:20.317481 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfef610-7013-4be5-838f-cd046ff18580 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:20.345318 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866a2b91-61cb-48ef-8f9c-200821f2661e None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:20.395656 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ce998f-5347-46c8-8593-ab272984af92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:20.395656 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ce998f-5347-46c8-8593-ab272984af92 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:20.395656 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ce998f-5347-46c8-8593-ab272984af92 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:20.395656 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ce998f-5347-46c8-8593-ab272984af92 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:20.406658 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ce998f-5347-46c8-8593-ab272984af92 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 12:02:20.416184 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:20.416540 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-52ce998f-5347-46c8-8593-ab272984af92 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 12:02:20.416540 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:02:20.416540 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:02:20.416540 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:02:20.416540 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:02:20.416540 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:02:20.416540 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:02:20.416540 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:02:20.416540 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:02:20.416540 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:02:20.416540 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:02:20.416540 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:02:20.416540 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 12:02:20.416540 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 12:02:20.416540 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 12:02:20.416540 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 12:02:20.417423 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 12:02:20.417423 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 12:02:20.417423 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 12:02:20.417423 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 12:02:20.417423 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:02:20.417423 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:02:20.417423 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:02:20.417423 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:02:20.417423 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:02:20.417423 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:02:20.417423 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:02:20.417423 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jul 25 12:02:20.417423 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:02:20.417423 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:02:20.417423 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:02:20.417423 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:02:20.418252 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:02:20.418252 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:02:20.418252 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:02:20.418252 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:02:20.418252 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 12:02:20.418252 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 12:02:20.418252 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 12:02:20.418252 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 12:02:20.418252 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 12:02:20.418252 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 12:02:20.418252 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 12:02:20.418252 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jul 25 12:02:20.418874 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 373/752] 10.222.0.42 () {56 vars in 1096 bytes} [Mon Jul 25 12:02:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:02:20.419287 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ce998f-5347-46c8-8593-ab272984af92 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:20.445495 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfef610-7013-4be5-838f-cd046ff18580 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:20.458712 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:20.481286 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21dc83f9-8849-42cc-a8b6-cfe6b9f03535 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:20.482114 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dc83f9-8849-42cc-a8b6-cfe6b9f03535 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:20.482396 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dc83f9-8849-42cc-a8b6-cfe6b9f03535 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:20.482671 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dc83f9-8849-42cc-a8b6-cfe6b9f03535 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:20.490315 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21dc83f9-8849-42cc-a8b6-cfe6b9f03535 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:20.491777 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21dc83f9-8849-42cc-a8b6-cfe6b9f03535 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:20.512637 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-21dc83f9-8849-42cc-a8b6-cfe6b9f03535 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 12:02:20.512637 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:02:20.512637 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:02:20.512637 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:02:20.512637 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:02:20.512637 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:02:20.512637 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:02:20.512637 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:02:20.512637 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:02:20.512637 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:02:20.512637 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:02:20.512637 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:02:20.512637 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 12:02:20.512637 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 12:02:20.512637 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 12:02:20.515237 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 12:02:20.515237 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:02:20.515237 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:02:20.515237 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:02:20.515237 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:02:20.515237 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:02:20.515237 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:02:20.515237 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:02:20.515237 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 25 12:02:20.515237 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:02:20.515237 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:02:20.515237 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:02:20.515237 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:02:20.515237 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:02:20.515237 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:02:20.515237 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:02:20.515237 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:02:20.517530 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 12:02:20.517530 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 12:02:20.517530 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 12:02:20.517530 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 12:02:20.517530 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 12:02:20.517530 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 12:02:20.517530 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 12:02:20.517530 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 25 12:02:20.517530 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 380/753] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Jul 25 12:02:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:02:20.517530 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21dc83f9-8849-42cc-a8b6-cfe6b9f03535 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:20.539955 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ce998f-5347-46c8-8593-ab272984af92 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:20.577460 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3e6b03-f7b1-4093-9c0e-d3c8ac51b3b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:20.578129 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3e6b03-f7b1-4093-9c0e-d3c8ac51b3b9 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:20.578421 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3e6b03-f7b1-4093-9c0e-d3c8ac51b3b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:20.578708 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3e6b03-f7b1-4093-9c0e-d3c8ac51b3b9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:20.579619 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3e6b03-f7b1-4093-9c0e-d3c8ac51b3b9 None admin] RBAC: Adding query filter params (role.id=79dec36aee5b4322bf652041c25ca6e4, scope.project.id=b9f62b8f37304b8a9a4bb54205347577, user.id=93207b858d294d049fa56d45810a40aa) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:02:20.579948 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3e6b03-f7b1-4093-9c0e-d3c8ac51b3b9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:20.581439 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3e6b03-f7b1-4093-9c0e-d3c8ac51b3b9 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:20.587480 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 374/754] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Jul 25 12:02:20 2022] GET /identity/v3/role_assignments?user.id=93207b858d294d049fa56d45810a40aa&scope.project.id=b9f62b8f37304b8a9a4bb54205347577&role.id=79dec36aee5b4322bf652041c25ca6e4 => generated 259 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:02:20.588295 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3e6b03-f7b1-4093-9c0e-d3c8ac51b3b9 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:20.678677 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21dc83f9-8849-42cc-a8b6-cfe6b9f03535 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:20.692619 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:20.706649 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3e6b03-f7b1-4093-9c0e-d3c8ac51b3b9 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:20.746062 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9aa009e-b654-4be4-999b-36bee68f3178 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:20.746765 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9aa009e-b654-4be4-999b-36bee68f3178 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:20.747065 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9aa009e-b654-4be4-999b-36bee68f3178 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:20.747361 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9aa009e-b654-4be4-999b-36bee68f3178 None admin] PATH_INFO: `/v3/projects` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:20.748217 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9aa009e-b654-4be4-999b-36bee68f3178 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:02:20.748546 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9aa009e-b654-4be4-999b-36bee68f3178 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:20.750399 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9aa009e-b654-4be4-999b-36bee68f3178 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:20.764700 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 375/755] 10.222.0.42 () {56 vars in 1148 bytes} [Mon Jul 25 12:02:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:02:20.785186 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9aa009e-b654-4be4-999b-36bee68f3178 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:20.793866 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b018e41-f4c8-4c7d-9b58-7fcfb46ed888 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:20.794561 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b018e41-f4c8-4c7d-9b58-7fcfb46ed888 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:20.794850 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b018e41-f4c8-4c7d-9b58-7fcfb46ed888 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:20.795133 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b018e41-f4c8-4c7d-9b58-7fcfb46ed888 None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:20.795935 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b018e41-f4c8-4c7d-9b58-7fcfb46ed888 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:02:20.796247 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b018e41-f4c8-4c7d-9b58-7fcfb46ed888 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:20.797636 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b018e41-f4c8-4c7d-9b58-7fcfb46ed888 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:20.805688 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:20.816668 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 381/756] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Jul 25 12:02:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:02:20.824271 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b018e41-f4c8-4c7d-9b58-7fcfb46ed888 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:20.921940 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9aa009e-b654-4be4-999b-36bee68f3178 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:20.933394 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b018e41-f4c8-4c7d-9b58-7fcfb46ed888 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:20.966619 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d332be5d-34b1-4bc4-b62b-bfec8f99c96d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:20.967496 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d332be5d-34b1-4bc4-b62b-bfec8f99c96d None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:20.967817 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d332be5d-34b1-4bc4-b62b-bfec8f99c96d None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:20.968204 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d332be5d-34b1-4bc4-b62b-bfec8f99c96d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:20.968701 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b239ef4f-6849-43a6-a355-39d6f2d2bdb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:20.969050 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d332be5d-34b1-4bc4-b62b-bfec8f99c96d None admin] RBAC: Adding query filter params (role.id=a7d20885b6d94928a34b3cc5b3a00006, scope.project.id=b9f62b8f37304b8a9a4bb54205347577, user.id=7bd782ea66364eeb96592096eae63d4f) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:02:20.969379 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d332be5d-34b1-4bc4-b62b-bfec8f99c96d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:20.969631 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b239ef4f-6849-43a6-a355-39d6f2d2bdb3 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:20.970550 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b239ef4f-6849-43a6-a355-39d6f2d2bdb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:20.970925 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b239ef4f-6849-43a6-a355-39d6f2d2bdb3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:20.971575 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d332be5d-34b1-4bc4-b62b-bfec8f99c96d None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:20.981850 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b239ef4f-6849-43a6-a355-39d6f2d2bdb3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:20.983458 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b239ef4f-6849-43a6-a355-39d6f2d2bdb3 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:20.987211 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 376/757] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Jul 25 12:02:20 2022] GET /identity/v3/role_assignments?user.id=7bd782ea66364eeb96592096eae63d4f&scope.project.id=b9f62b8f37304b8a9a4bb54205347577&role.id=a7d20885b6d94928a34b3cc5b3a00006 => generated 259 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:02:21.015973 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-b239ef4f-6849-43a6-a355-39d6f2d2bdb3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:02:21.015973 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:02:21.015973 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:02:21.015973 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:02:21.015973 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:02:21.015973 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:02:21.015973 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:02:21.015973 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:02:21.015973 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:02:21.015973 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:02:21.015973 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:02:21.015973 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:02:21.015973 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 12:02:21.015973 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 12:02:21.015973 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 12:02:21.017463 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 12:02:21.017463 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:02:21.017463 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:02:21.017463 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:02:21.017463 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:02:21.017463 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:02:21.017463 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:02:21.017463 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:02:21.017463 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 25 12:02:21.017463 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:02:21.017463 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:02:21.017463 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:02:21.017463 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:02:21.017463 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:02:21.017463 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:02:21.017463 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:02:21.017463 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:02:21.018347 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 12:02:21.018347 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 12:02:21.018347 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:02:21.018347 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 25 12:02:21.018347 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 382/758] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jul 25 12:02:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:02:21.023845 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d332be5d-34b1-4bc4-b62b-bfec8f99c96d None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:21.046564 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:21.255130 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d332be5d-34b1-4bc4-b62b-bfec8f99c96d None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:21.290272 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcedd70-31da-4341-b94b-db2631e90c20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:21.291206 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcedd70-31da-4341-b94b-db2631e90c20 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:21.291495 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcedd70-31da-4341-b94b-db2631e90c20 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:21.291811 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcedd70-31da-4341-b94b-db2631e90c20 None admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:21.292549 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afcedd70-31da-4341-b94b-db2631e90c20 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:02:21.292869 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afcedd70-31da-4341-b94b-db2631e90c20 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:21.294249 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afcedd70-31da-4341-b94b-db2631e90c20 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:21.306972 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 377/759] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jul 25 12:02:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:02:21.312450 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b239ef4f-6849-43a6-a355-39d6f2d2bdb3 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:21.394685 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:21.427655 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8204017e-93f1-490b-9102-488bdadb2346 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:21.427971 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8204017e-93f1-490b-9102-488bdadb2346 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:21.428228 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8204017e-93f1-490b-9102-488bdadb2346 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:21.429826 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 378/760] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 25 12:02:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:02:21.435138 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf614b5-b8b5-4fa1-8474-5ba5acef1258 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:21.435720 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf614b5-b8b5-4fa1-8474-5ba5acef1258 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:21.436243 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf614b5-b8b5-4fa1-8474-5ba5acef1258 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:21.456047 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-ebf614b5-b8b5-4fa1-8474-5ba5acef1258 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:02:21.456367 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b239ef4f-6849-43a6-a355-39d6f2d2bdb3 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:21.457916 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-ebf614b5-b8b5-4fa1-8474-5ba5acef1258 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:02:21.499217 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed0f436-0c3d-4289-8759-7c888c669a02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:21.500108 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed0f436-0c3d-4289-8759-7c888c669a02 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:21.500399 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed0f436-0c3d-4289-8759-7c888c669a02 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:21.500684 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed0f436-0c3d-4289-8759-7c888c669a02 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:21.514552 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed0f436-0c3d-4289-8759-7c888c669a02 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:21.517524 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed0f436-0c3d-4289-8759-7c888c669a02 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:21.544843 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-bed0f436-0c3d-4289-8759-7c888c669a02 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 25 12:02:21.544843 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:02:21.544843 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:02:21.544843 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:02:21.544843 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:02:21.544843 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:02:21.544843 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:02:21.544843 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:02:21.544843 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:02:21.544843 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:02:21.544843 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:02:21.544843 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:02:21.544843 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 12:02:21.544843 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 12:02:21.544843 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 12:02:21.544843 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 12:02:21.545849 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:02:21.545849 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:02:21.545849 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 12:02:21.545849 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 12:02:21.545849 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 12:02:21.545849 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 12:02:21.545849 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:02:21.545849 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:02:21.545849 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:02:21.545849 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:02:21.545849 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:02:21.545849 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 25 12:02:21.545849 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:02:21.545849 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:02:21.545849 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:02:21.545849 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:02:21.545849 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:02:21.545849 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:02:21.546832 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:02:21.546832 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:02:21.546832 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 12:02:21.546832 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 12:02:21.546832 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 12:02:21.546832 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 12:02:21.546832 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 12:02:21.546832 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 12:02:21.546832 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 25 12:02:21.546832 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 25 12:02:21.547365 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 383/761] 10.222.0.42 () {56 vars in 1093 bytes} [Mon Jul 25 12:02:21 2022] GET /identity/v3/users/nova => generated 82 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:02:21.559663 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed0f436-0c3d-4289-8759-7c888c669a02 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:21.589637 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 379/762] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 25 12:02:21 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 25 12:02:21.624311 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf614b5-b8b5-4fa1-8474-5ba5acef1258 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:21.640067 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:21.663029 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed0f436-0c3d-4289-8759-7c888c669a02 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:21.692020 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:21.703950 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd58f8f2-9785-4056-807c-0a2b5dc14ed3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:21.705625 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd58f8f2-9785-4056-807c-0a2b5dc14ed3 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:21.706855 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd58f8f2-9785-4056-807c-0a2b5dc14ed3 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:21.707344 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd58f8f2-9785-4056-807c-0a2b5dc14ed3 None admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:21.708405 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd58f8f2-9785-4056-807c-0a2b5dc14ed3 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:02:21.708903 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd58f8f2-9785-4056-807c-0a2b5dc14ed3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:21.711988 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd58f8f2-9785-4056-807c-0a2b5dc14ed3 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:21.755265 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 384/763] 10.222.0.42 () {56 vars in 1133 bytes} [Mon Jul 25 12:02:21 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 12:02:21.756452 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd58f8f2-9785-4056-807c-0a2b5dc14ed3 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:21.790101 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf614b5-b8b5-4fa1-8474-5ba5acef1258 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:21.827781 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071e740e-183e-4384-a3cf-9e4c03ed0042 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:21.828867 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071e740e-183e-4384-a3cf-9e4c03ed0042 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:21.829325 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071e740e-183e-4384-a3cf-9e4c03ed0042 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:21.829782 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071e740e-183e-4384-a3cf-9e4c03ed0042 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:21.830587 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071e740e-183e-4384-a3cf-9e4c03ed0042 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:21.832124 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071e740e-183e-4384-a3cf-9e4c03ed0042 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:21.847309 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-071e740e-183e-4384-a3cf-9e4c03ed0042 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 25 12:02:21.847309 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:02:21.847309 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 12:02:21.847309 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 12:02:21.847309 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 12:02:21.847309 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 12:02:21.847309 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 12:02:21.847309 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 12:02:21.847309 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 25 12:02:21.847309 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jul 25 12:02:21.847309 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 12:02:21.847309 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jul 25 12:02:21.847309 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:02:21.847309 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:02:21.847309 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:02:21.847309 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:02:21.848490 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:02:21.848490 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:02:21.848490 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:02:21.848490 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:02:21.848490 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:02:21.848490 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:02:21.848490 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:02:21.848490 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 12:02:21.848490 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 12:02:21.848490 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 12:02:21.848490 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 12:02:21.848490 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:02:21.848490 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:02:21.848490 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:02:21.848490 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:02:21.848490 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:02:21.849434 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:02:21.849434 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:02:21.849434 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jul 25 12:02:21.849434 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:02:21.849434 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:02:21.849434 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:02:21.849434 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:02:21.849434 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:02:21.849434 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:02:21.849434 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:02:21.849434 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:02:21.849434 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 12:02:21.849434 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 12:02:21.849434 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 25 12:02:21.849434 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jul 25 12:02:21.852035 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 380/764] 10.222.0.42 () {56 vars in 1129 bytes} [Mon Jul 25 12:02:21 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 12:02:21.857775 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071e740e-183e-4384-a3cf-9e4c03ed0042 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:21.967477 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd58f8f2-9785-4056-807c-0a2b5dc14ed3 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:21.987871 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:22.045700 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433efc45-0b0f-46c7-9de8-173ebdeed9cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:22.047619 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433efc45-0b0f-46c7-9de8-173ebdeed9cb None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:22.049432 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433efc45-0b0f-46c7-9de8-173ebdeed9cb None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:22.051452 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071e740e-183e-4384-a3cf-9e4c03ed0042 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:22.052003 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433efc45-0b0f-46c7-9de8-173ebdeed9cb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:22.072452 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433efc45-0b0f-46c7-9de8-173ebdeed9cb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:22.076357 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433efc45-0b0f-46c7-9de8-173ebdeed9cb None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:22.109201 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-433efc45-0b0f-46c7-9de8-173ebdeed9cb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:02:22.109201 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:02:22.109201 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:02:22.109201 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:02:22.109201 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:02:22.109201 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:02:22.109201 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:02:22.109201 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:02:22.109201 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:02:22.109201 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:02:22.109201 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:02:22.109201 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:02:22.109201 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 12:02:22.109201 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 12:02:22.109201 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 12:02:22.110156 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 12:02:22.110156 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:02:22.110156 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:02:22.110156 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:02:22.110156 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:02:22.110156 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:02:22.110156 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:02:22.110156 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:02:22.110156 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 25 12:02:22.110156 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:02:22.110156 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:02:22.110156 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:02:22.110156 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:02:22.110156 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:02:22.110156 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:02:22.110156 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:02:22.110156 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:02:22.111066 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 12:02:22.111066 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 12:02:22.111066 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:02:22.111066 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 25 12:02:22.113535 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 385/765] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jul 25 12:02:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 357 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:02:22.119495 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433efc45-0b0f-46c7-9de8-173ebdeed9cb None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:22.138121 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0167c28-a62f-4040-a455-eadaabfadf12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:22.139046 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0167c28-a62f-4040-a455-eadaabfadf12 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:22.139507 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0167c28-a62f-4040-a455-eadaabfadf12 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:22.139951 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0167c28-a62f-4040-a455-eadaabfadf12 None admin] PATH_INFO: `/v3/services` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:22.140824 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0167c28-a62f-4040-a455-eadaabfadf12 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:02:22.141290 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0167c28-a62f-4040-a455-eadaabfadf12 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:22.143354 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0167c28-a62f-4040-a455-eadaabfadf12 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:22.155512 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 381/766] 10.222.0.42 () {56 vars in 1124 bytes} [Mon Jul 25 12:02:21 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:02:22.163670 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0167c28-a62f-4040-a455-eadaabfadf12 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:22.163821 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:22.284141 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433efc45-0b0f-46c7-9de8-173ebdeed9cb None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:22.326165 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6272329b-fd3f-4c80-92d7-d3d12b3a2ce3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:22.327161 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6272329b-fd3f-4c80-92d7-d3d12b3a2ce3 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:22.327458 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6272329b-fd3f-4c80-92d7-d3d12b3a2ce3 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:22.327773 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6272329b-fd3f-4c80-92d7-d3d12b3a2ce3 None admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:22.328513 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6272329b-fd3f-4c80-92d7-d3d12b3a2ce3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:02:22.328832 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6272329b-fd3f-4c80-92d7-d3d12b3a2ce3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:22.330778 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6272329b-fd3f-4c80-92d7-d3d12b3a2ce3 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:22.344883 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 386/767] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jul 25 12:02:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 12:02:22.353657 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6272329b-fd3f-4c80-92d7-d3d12b3a2ce3 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:22.358089 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0167c28-a62f-4040-a455-eadaabfadf12 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:22.444771 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:22.452248 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a800be-ac60-41bb-9cb2-2856b74870ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:22.452937 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a800be-ac60-41bb-9cb2-2856b74870ce None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:22.453226 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a800be-ac60-41bb-9cb2-2856b74870ce None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:22.453598 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a800be-ac60-41bb-9cb2-2856b74870ce None admin] PATH_INFO: `/v3/services` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:22.454543 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a800be-ac60-41bb-9cb2-2856b74870ce None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:02:22.454543 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a800be-ac60-41bb-9cb2-2856b74870ce None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:22.456122 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a800be-ac60-41bb-9cb2-2856b74870ce None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:22.457724 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:22.466999 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 382/768] 10.222.0.42 () {56 vars in 1124 bytes} [Mon Jul 25 12:02:22 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:02:22.468233 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6272329b-fd3f-4c80-92d7-d3d12b3a2ce3 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:22.475660 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a800be-ac60-41bb-9cb2-2856b74870ce None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:22.508725 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b870bf7-df91-437d-897f-cd7726c8b454 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:22.509891 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b870bf7-df91-437d-897f-cd7726c8b454 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:22.510539 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b870bf7-df91-437d-897f-cd7726c8b454 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:22.512294 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b870bf7-df91-437d-897f-cd7726c8b454 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:22.523405 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b870bf7-df91-437d-897f-cd7726c8b454 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:22.525245 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b870bf7-df91-437d-897f-cd7726c8b454 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:22.540541 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-5b870bf7-df91-437d-897f-cd7726c8b454 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 12:02:22.540541 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:02:22.540541 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:02:22.540541 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:02:22.540541 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:02:22.540541 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:02:22.540541 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:02:22.540541 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:02:22.540541 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:02:22.540541 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:02:22.540541 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:02:22.540541 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:02:22.540541 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 12:02:22.540541 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 12:02:22.540541 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 12:02:22.541675 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 12:02:22.541675 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:02:22.541675 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:02:22.541675 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:02:22.541675 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:02:22.541675 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:02:22.541675 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:02:22.541675 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:02:22.541675 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 25 12:02:22.541675 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:02:22.541675 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:02:22.541675 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:02:22.541675 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:02:22.541675 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:02:22.541675 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:02:22.541675 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:02:22.541675 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:02:22.542567 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 12:02:22.542567 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 12:02:22.542567 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 12:02:22.542567 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 12:02:22.542567 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 12:02:22.542567 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 12:02:22.542567 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 12:02:22.542567 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 25 12:02:22.544957 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 387/769] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Jul 25 12:02:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:02:22.546365 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac16a08-196c-4a9c-94f8-7cbbb936d2b1 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:22.551570 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac16a08-196c-4a9c-94f8-7cbbb936d2b1 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:22.552108 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac16a08-196c-4a9c-94f8-7cbbb936d2b1 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:22.554091 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 388/770] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 25 12:02:22 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:02:22.555753 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac16a08-196c-4a9c-94f8-7cbbb936d2b1 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:22.614538 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a800be-ac60-41bb-9cb2-2856b74870ce None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:22.655681 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae63617-c2ef-4c8b-8e10-422bf1226543 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:22.657199 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae63617-c2ef-4c8b-8e10-422bf1226543 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:22.657738 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae63617-c2ef-4c8b-8e10-422bf1226543 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:22.658630 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae63617-c2ef-4c8b-8e10-422bf1226543 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:22.659960 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae63617-c2ef-4c8b-8e10-422bf1226543 None admin] RBAC: Adding query filter params (interface=public, service_id=0e3e145b16c348bc910a023837847ed9, region_id=RegionOne) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:02:22.660485 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae63617-c2ef-4c8b-8e10-422bf1226543 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:22.662571 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae63617-c2ef-4c8b-8e10-422bf1226543 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:22.672599 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 383/771] 10.222.0.42 () {56 vars in 1251 bytes} [Mon Jul 25 12:02:22 2022] GET /identity/v3/endpoints?service_id=0e3e145b16c348bc910a023837847ed9&interface=public®ion_id=RegionOne => generated 194 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 12:02:22.674630 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2945e57f-6f6b-4eaa-ab9f-05e82e1525c5 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:22.675126 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2945e57f-6f6b-4eaa-ab9f-05e82e1525c5 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:22.675679 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2945e57f-6f6b-4eaa-ab9f-05e82e1525c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:22.689690 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac16a08-196c-4a9c-94f8-7cbbb936d2b1 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:22.697850 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-2945e57f-6f6b-4eaa-ab9f-05e82e1525c5 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:02:22.700006 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-2945e57f-6f6b-4eaa-ab9f-05e82e1525c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:02:22.736607 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33be038c-c710-4002-a763-2aa4d464503d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:22.737641 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33be038c-c710-4002-a763-2aa4d464503d None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:22.738136 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33be038c-c710-4002-a763-2aa4d464503d None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:22.738643 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33be038c-c710-4002-a763-2aa4d464503d None admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:22.739801 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33be038c-c710-4002-a763-2aa4d464503d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:02:22.740314 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33be038c-c710-4002-a763-2aa4d464503d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:22.742155 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33be038c-c710-4002-a763-2aa4d464503d None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:22.760937 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 389/772] 10.222.0.42 () {56 vars in 1148 bytes} [Mon Jul 25 12:02:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:02:22.762291 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33be038c-c710-4002-a763-2aa4d464503d None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:22.765371 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:22.773878 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:22.789315 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 384/773] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 25 12:02:22 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 12:02:22.790566 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2945e57f-6f6b-4eaa-ab9f-05e82e1525c5 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:22.919295 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33be038c-c710-4002-a763-2aa4d464503d None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:22.923742 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2945e57f-6f6b-4eaa-ab9f-05e82e1525c5 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:22.954675 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12223059-5379-4b18-860d-25b378c7abe5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:22.955817 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12223059-5379-4b18-860d-25b378c7abe5 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:22.956301 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12223059-5379-4b18-860d-25b378c7abe5 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:22.956799 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12223059-5379-4b18-860d-25b378c7abe5 None admin] PATH_INFO: `/v3/services` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:22.957673 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12223059-5379-4b18-860d-25b378c7abe5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:22.959262 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12223059-5379-4b18-860d-25b378c7abe5 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:22.965918 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 390/774] 10.222.0.42 () {56 vars in 1087 bytes} [Mon Jul 25 12:02:22 2022] GET /identity/v3/services => generated 772 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:02:22.966925 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12223059-5379-4b18-860d-25b378c7abe5 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:22.974159 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c4a4d3-450e-4839-9f7c-96240b0cb7e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:22.977402 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c4a4d3-450e-4839-9f7c-96240b0cb7e1 None admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:22.977998 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c4a4d3-450e-4839-9f7c-96240b0cb7e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:22.979707 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c4a4d3-450e-4839-9f7c-96240b0cb7e1 None admin] PATH_INFO: `/v3/projects/b9f62b8f37304b8a9a4bb54205347577/users/f67142a1193d47c7bd33d7040268233c/roles/a7d20885b6d94928a34b3cc5b3a00006` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:22.984233 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c4a4d3-450e-4839-9f7c-96240b0cb7e1 None admin] RBAC: Authorizing `identity:create_grant(project_id=b9f62b8f37304b8a9a4bb54205347577, user_id=f67142a1193d47c7bd33d7040268233c, role_id=a7d20885b6d94928a34b3cc5b3a00006)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:22.986763 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c4a4d3-450e-4839-9f7c-96240b0cb7e1 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:23.012239 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 385/775] 10.222.0.42 () {58 vars in 1439 bytes} [Mon Jul 25 12:02:22 2022] PUT /identity/v3/projects/b9f62b8f37304b8a9a4bb54205347577/users/f67142a1193d47c7bd33d7040268233c/roles/a7d20885b6d94928a34b3cc5b3a00006 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:02:23.065425 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:23.121988 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12223059-5379-4b18-860d-25b378c7abe5 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:23.175786 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79f7251-cbcb-4def-9ad9-ac0fbe7a3354 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:23.176987 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79f7251-cbcb-4def-9ad9-ac0fbe7a3354 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:23.177483 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79f7251-cbcb-4def-9ad9-ac0fbe7a3354 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:23.177957 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79f7251-cbcb-4def-9ad9-ac0fbe7a3354 None admin] PATH_INFO: `/v3/services/network` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:23.178833 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79f7251-cbcb-4def-9ad9-ac0fbe7a3354 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:23.180492 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79f7251-cbcb-4def-9ad9-ac0fbe7a3354 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:23.194130 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-d79f7251-cbcb-4def-9ad9-ac0fbe7a3354 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 25 12:02:23.194130 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:02:23.194130 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 12:02:23.194130 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 12:02:23.194130 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 12:02:23.194130 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 12:02:23.194130 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 12:02:23.194130 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 12:02:23.194130 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 25 12:02:23.194130 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 25 12:02:23.194130 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 12:02:23.194130 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 25 12:02:23.194130 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:02:23.194130 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:02:23.194130 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:02:23.194130 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:02:23.194130 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:02:23.195175 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:02:23.195175 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:02:23.195175 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:02:23.195175 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:02:23.195175 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:02:23.195175 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:02:23.195175 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 12:02:23.195175 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 12:02:23.195175 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 12:02:23.195175 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 12:02:23.195175 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:02:23.195175 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:02:23.195175 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:02:23.195175 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:02:23.195175 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:02:23.195175 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:02:23.196054 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:02:23.196054 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 25 12:02:23.196054 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:02:23.196054 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:02:23.196054 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:02:23.196054 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:02:23.196054 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:02:23.196054 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:02:23.196054 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:02:23.196054 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:02:23.196054 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 12:02:23.196054 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 12:02:23.196054 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 25 12:02:23.196054 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 25 12:02:23.199023 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 391/776] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Jul 25 12:02:22 2022] GET /identity/v3/services/network => generated 88 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:02:23.207647 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c4a4d3-450e-4839-9f7c-96240b0cb7e1 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:23.207951 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9a0c2d-bfd6-4d99-bb8a-72bb97efdded None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:23.208238 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9a0c2d-bfd6-4d99-bb8a-72bb97efdded None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:23.208522 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9a0c2d-bfd6-4d99-bb8a-72bb97efdded None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:23.210051 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 392/777] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 25 12:02:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:02:23.215713 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacf4169-1333-4507-aa49-5c2e69c5b6d4 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:23.216317 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacf4169-1333-4507-aa49-5c2e69c5b6d4 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:23.217979 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacf4169-1333-4507-aa49-5c2e69c5b6d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:23.242531 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-bacf4169-1333-4507-aa49-5c2e69c5b6d4 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:02:23.244395 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-bacf4169-1333-4507-aa49-5c2e69c5b6d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:02:23.279336 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:23.340840 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 393/778] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 25 12:02:23 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 12:02:23.342190 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4aa7ed7-f52b-4c2b-93f2-b81aed66e410 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:23.342478 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4aa7ed7-f52b-4c2b-93f2-b81aed66e410 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:23.342753 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4aa7ed7-f52b-4c2b-93f2-b81aed66e410 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:23.344081 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 394/779] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 25 12:02:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:02:23.349969 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef8fb09-9164-40f6-ac47-57198eb9e6c2 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:23.350257 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef8fb09-9164-40f6-ac47-57198eb9e6c2 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:23.350514 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c4a4d3-450e-4839-9f7c-96240b0cb7e1 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:23.350634 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef8fb09-9164-40f6-ac47-57198eb9e6c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:23.368476 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-3ef8fb09-9164-40f6-ac47-57198eb9e6c2 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:02:23.369768 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-3ef8fb09-9164-40f6-ac47-57198eb9e6c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:02:23.393097 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e23063-9f59-4e3d-a53b-63f27423e20f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:23.393898 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e23063-9f59-4e3d-a53b-63f27423e20f None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:23.394189 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e23063-9f59-4e3d-a53b-63f27423e20f None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:23.394585 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e23063-9f59-4e3d-a53b-63f27423e20f None admin] PATH_INFO: `/v3/services` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:23.395307 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e23063-9f59-4e3d-a53b-63f27423e20f None admin] RBAC: Adding query filter params (name=network) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:02:23.395655 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e23063-9f59-4e3d-a53b-63f27423e20f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:23.397138 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e23063-9f59-4e3d-a53b-63f27423e20f None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:23.402886 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 386/780] 10.222.0.42 () {56 vars in 1112 bytes} [Mon Jul 25 12:02:23 2022] GET /identity/v3/services?name=network => generated 124 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:02:23.404617 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e23063-9f59-4e3d-a53b-63f27423e20f None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:23.455108 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:23.474630 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 395/781] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 25 12:02:23 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 25 12:02:23.479615 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef8fb09-9164-40f6-ac47-57198eb9e6c2 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:23.547736 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e23063-9f59-4e3d-a53b-63f27423e20f None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:23.568822 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:23.589712 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a417866-769c-4e06-be88-9f39dea6c9f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:23.590832 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a417866-769c-4e06-be88-9f39dea6c9f3 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:23.591319 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a417866-769c-4e06-be88-9f39dea6c9f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:23.591808 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a417866-769c-4e06-be88-9f39dea6c9f3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:23.595965 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef8fb09-9164-40f6-ac47-57198eb9e6c2 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:23.603920 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a417866-769c-4e06-be88-9f39dea6c9f3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 12:02:23.612394 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-8a417866-769c-4e06-be88-9f39dea6c9f3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 12:02:23.612394 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:02:23.612394 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:02:23.612394 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:02:23.612394 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:02:23.612394 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:02:23.612394 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:02:23.612394 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:02:23.612394 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:02:23.612394 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:02:23.612394 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:02:23.612394 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:02:23.612394 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 12:02:23.612394 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 12:02:23.612394 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 12:02:23.612394 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 12:02:23.613310 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 12:02:23.613310 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 12:02:23.613310 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 12:02:23.613310 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 12:02:23.613310 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:02:23.613310 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:02:23.613310 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:02:23.613310 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:02:23.613310 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:02:23.613310 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:02:23.613310 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:02:23.613310 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jul 25 12:02:23.613310 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:02:23.613310 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:02:23.613310 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:02:23.613310 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:02:23.614123 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:02:23.614123 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:02:23.614123 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:02:23.614123 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:02:23.614123 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 12:02:23.614123 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 12:02:23.614123 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 12:02:23.614123 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 12:02:23.614123 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 12:02:23.614123 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 12:02:23.614123 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 12:02:23.614123 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jul 25 12:02:23.616861 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 387/782] 10.222.0.42 () {56 vars in 1096 bytes} [Mon Jul 25 12:02:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:02:23.617972 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a417866-769c-4e06-be88-9f39dea6c9f3 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:23.635049 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345149b0-4345-47e1-9ab1-12cdd44ba7a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:23.635859 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345149b0-4345-47e1-9ab1-12cdd44ba7a3 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:23.637144 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345149b0-4345-47e1-9ab1-12cdd44ba7a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:23.637621 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345149b0-4345-47e1-9ab1-12cdd44ba7a3 None admin] PATH_INFO: `/v3/services` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:23.638692 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345149b0-4345-47e1-9ab1-12cdd44ba7a3 None admin] RBAC: Adding query filter params (type=network) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:02:23.639937 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345149b0-4345-47e1-9ab1-12cdd44ba7a3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:23.641851 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345149b0-4345-47e1-9ab1-12cdd44ba7a3 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:23.652495 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 396/783] 10.222.0.42 () {56 vars in 1112 bytes} [Mon Jul 25 12:02:23 2022] GET /identity/v3/services?type=network => generated 350 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:02:23.652495 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345149b0-4345-47e1-9ab1-12cdd44ba7a3 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:23.715098 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:23.738271 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a417866-769c-4e06-be88-9f39dea6c9f3 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:23.765264 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345149b0-4345-47e1-9ab1-12cdd44ba7a3 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:23.776076 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa158fc-e472-4e86-9849-58792fba910d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:23.777273 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa158fc-e472-4e86-9849-58792fba910d None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:23.777856 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa158fc-e472-4e86-9849-58792fba910d None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:23.778336 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa158fc-e472-4e86-9849-58792fba910d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:23.791607 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa158fc-e472-4e86-9849-58792fba910d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 12:02:23.801308 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-5fa158fc-e472-4e86-9849-58792fba910d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 25 12:02:23.801308 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:02:23.801308 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:02:23.801308 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:02:23.801308 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:02:23.801308 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:02:23.801308 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:02:23.801308 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:02:23.801308 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:02:23.801308 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:02:23.801308 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:02:23.801308 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:02:23.801308 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 12:02:23.801308 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 12:02:23.801308 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 12:02:23.801308 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 12:02:23.802433 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 12:02:23.802433 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 12:02:23.802433 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 12:02:23.802433 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 12:02:23.802433 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:02:23.802433 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:02:23.802433 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:02:23.802433 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:02:23.802433 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:02:23.802433 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:02:23.802433 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:02:23.802433 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jul 25 12:02:23.802433 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:02:23.802433 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:02:23.802433 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:02:23.802433 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:02:23.803467 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:02:23.803467 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:02:23.803467 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:02:23.803467 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:02:23.803467 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 12:02:23.803467 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 12:02:23.803467 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 12:02:23.803467 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 12:02:23.803467 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 12:02:23.803467 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 12:02:23.803467 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 25 12:02:23.803467 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jul 25 12:02:23.806279 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 388/784] 10.222.0.42 () {56 vars in 1102 bytes} [Mon Jul 25 12:02:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:02:23.807036 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979b7017-ab2f-425b-bc5f-57eb6dbf7816 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:23.807476 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa158fc-e472-4e86-9849-58792fba910d None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:23.807948 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979b7017-ab2f-425b-bc5f-57eb6dbf7816 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:23.808331 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979b7017-ab2f-425b-bc5f-57eb6dbf7816 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:23.808720 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979b7017-ab2f-425b-bc5f-57eb6dbf7816 None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:23.809569 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979b7017-ab2f-425b-bc5f-57eb6dbf7816 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:02:23.809985 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979b7017-ab2f-425b-bc5f-57eb6dbf7816 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:23.811730 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979b7017-ab2f-425b-bc5f-57eb6dbf7816 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:23.821145 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 397/785] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Jul 25 12:02:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:02:23.826314 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979b7017-ab2f-425b-bc5f-57eb6dbf7816 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:23.848094 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:23.925343 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa158fc-e472-4e86-9849-58792fba910d None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:23.964578 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798c2038-f6c2-4d2b-bec0-ac0b946ed297 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:23.965654 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798c2038-f6c2-4d2b-bec0-ac0b946ed297 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:23.966136 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798c2038-f6c2-4d2b-bec0-ac0b946ed297 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:23.966597 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798c2038-f6c2-4d2b-bec0-ac0b946ed297 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:23.967593 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798c2038-f6c2-4d2b-bec0-ac0b946ed297 None admin] RBAC: Adding query filter params (interface=public, service_id=7579445d656c4da08ba69db8e996fcb8, region_id=RegionOne) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:02:23.968152 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798c2038-f6c2-4d2b-bec0-ac0b946ed297 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:23.969858 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798c2038-f6c2-4d2b-bec0-ac0b946ed297 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:23.976041 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 389/786] 10.222.0.42 () {56 vars in 1251 bytes} [Mon Jul 25 12:02:23 2022] GET /identity/v3/endpoints?service_id=7579445d656c4da08ba69db8e996fcb8&interface=public®ion_id=RegionOne => generated 194 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:02:23.977010 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798c2038-f6c2-4d2b-bec0-ac0b946ed297 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:24.046676 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979b7017-ab2f-425b-bc5f-57eb6dbf7816 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:24.063694 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:24.085332 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:24.086899 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ffa8bd-8554-4736-b749-9fb9df2730be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:24.087740 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ffa8bd-8554-4736-b749-9fb9df2730be None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:24.088067 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ffa8bd-8554-4736-b749-9fb9df2730be None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:24.091795 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ffa8bd-8554-4736-b749-9fb9df2730be None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:24.091795 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ffa8bd-8554-4736-b749-9fb9df2730be None admin] RBAC: Adding query filter params (name=service) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:02:24.091795 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ffa8bd-8554-4736-b749-9fb9df2730be None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:24.092043 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798c2038-f6c2-4d2b-bec0-ac0b946ed297 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:24.092116 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ffa8bd-8554-4736-b749-9fb9df2730be None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:24.100328 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 398/787] 10.222.0.42 () {56 vars in 1103 bytes} [Mon Jul 25 12:02:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:02:24.101369 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ffa8bd-8554-4736-b749-9fb9df2730be None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:24.129931 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3c9c00-de1a-4066-9063-e5876a7de64b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:24.131023 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3c9c00-de1a-4066-9063-e5876a7de64b None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:24.131490 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3c9c00-de1a-4066-9063-e5876a7de64b None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:24.131967 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3c9c00-de1a-4066-9063-e5876a7de64b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:24.143185 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3c9c00-de1a-4066-9063-e5876a7de64b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:24.144958 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3c9c00-de1a-4066-9063-e5876a7de64b None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:24.163812 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-9a3c9c00-de1a-4066-9063-e5876a7de64b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:02:24.163812 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:02:24.163812 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:02:24.163812 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:02:24.163812 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:02:24.163812 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:02:24.163812 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:02:24.163812 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:02:24.163812 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:02:24.163812 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:02:24.163812 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:02:24.163812 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:02:24.163812 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 12:02:24.163812 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 12:02:24.163812 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 12:02:24.164723 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 12:02:24.164723 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:02:24.164723 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:02:24.164723 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:02:24.164723 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:02:24.164723 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:02:24.164723 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:02:24.164723 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:02:24.164723 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jul 25 12:02:24.164723 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:02:24.164723 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:02:24.164723 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:02:24.164723 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:02:24.164723 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:02:24.164723 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:02:24.164723 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:02:24.164723 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:02:24.165611 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 12:02:24.165611 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 12:02:24.165611 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:02:24.165611 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jul 25 12:02:24.168103 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 390/788] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jul 25 12:02:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:02:24.169049 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3c9c00-de1a-4066-9063-e5876a7de64b None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:24.209148 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ffa8bd-8554-4736-b749-9fb9df2730be None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:24.246431 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0322b3b5-053d-4173-8c19-f3f568228f8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:24.247116 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0322b3b5-053d-4173-8c19-f3f568228f8e None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:24.247476 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0322b3b5-053d-4173-8c19-f3f568228f8e None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:24.247860 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0322b3b5-053d-4173-8c19-f3f568228f8e None admin] PATH_INFO: `/v3/services` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:24.248646 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0322b3b5-053d-4173-8c19-f3f568228f8e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:24.250268 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0322b3b5-053d-4173-8c19-f3f568228f8e None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:24.256387 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 399/789] 10.222.0.42 () {56 vars in 1087 bytes} [Mon Jul 25 12:02:24 2022] GET /identity/v3/services => generated 772 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:02:24.257384 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0322b3b5-053d-4173-8c19-f3f568228f8e None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:24.285559 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3c9c00-de1a-4066-9063-e5876a7de64b None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:24.304620 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:24.327912 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79da10a2-8044-4644-9858-a9a088fde4b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:24.329136 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79da10a2-8044-4644-9858-a9a088fde4b7 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:24.330167 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79da10a2-8044-4644-9858-a9a088fde4b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:24.330722 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79da10a2-8044-4644-9858-a9a088fde4b7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:24.343299 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79da10a2-8044-4644-9858-a9a088fde4b7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:24.345693 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79da10a2-8044-4644-9858-a9a088fde4b7 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:24.365495 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-79da10a2-8044-4644-9858-a9a088fde4b7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:02:24.365495 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:02:24.365495 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:02:24.365495 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:02:24.365495 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:02:24.365495 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:02:24.365495 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:02:24.365495 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:02:24.365495 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:02:24.365495 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:02:24.365495 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:02:24.365495 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:02:24.365495 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 12:02:24.365495 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 12:02:24.365495 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 12:02:24.366547 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 12:02:24.366547 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:02:24.366547 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:02:24.366547 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:02:24.366547 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:02:24.366547 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:02:24.366547 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:02:24.366547 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:02:24.366547 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jul 25 12:02:24.366547 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:02:24.366547 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:02:24.366547 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:02:24.366547 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:02:24.366547 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:02:24.366547 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:02:24.366547 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:02:24.366547 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:02:24.367431 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 12:02:24.367431 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 12:02:24.367431 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:02:24.367431 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jul 25 12:02:24.367692 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0322b3b5-053d-4173-8c19-f3f568228f8e None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:24.370148 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 391/790] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jul 25 12:02:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:02:24.380748 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79da10a2-8044-4644-9858-a9a088fde4b7 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:24.385117 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:24.405604 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5780efde-26be-4950-8eb7-be1ca8037fd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:24.406282 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5780efde-26be-4950-8eb7-be1ca8037fd9 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:24.406572 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5780efde-26be-4950-8eb7-be1ca8037fd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:24.406856 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5780efde-26be-4950-8eb7-be1ca8037fd9 None admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:24.407673 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5780efde-26be-4950-8eb7-be1ca8037fd9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:02:24.407987 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5780efde-26be-4950-8eb7-be1ca8037fd9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:24.409518 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5780efde-26be-4950-8eb7-be1ca8037fd9 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:24.426608 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 400/791] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jul 25 12:02:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:02:24.427962 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5780efde-26be-4950-8eb7-be1ca8037fd9 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:24.535628 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5780efde-26be-4950-8eb7-be1ca8037fd9 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:24.537254 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:24.573475 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ffde12-9cf8-4789-be0a-694be7112419 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:24.574449 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ffde12-9cf8-4789-be0a-694be7112419 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:24.574741 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ffde12-9cf8-4789-be0a-694be7112419 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:24.575130 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ffde12-9cf8-4789-be0a-694be7112419 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:24.582392 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79da10a2-8044-4644-9858-a9a088fde4b7 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:24.598250 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ffde12-9cf8-4789-be0a-694be7112419 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:24.599760 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ffde12-9cf8-4789-be0a-694be7112419 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:24.615924 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-e4ffde12-9cf8-4789-be0a-694be7112419 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 25 12:02:24.615924 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:02:24.615924 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:02:24.615924 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:02:24.615924 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:02:24.615924 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:02:24.615924 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:02:24.615924 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:02:24.615924 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:02:24.615924 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:02:24.615924 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:02:24.615924 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:02:24.615924 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 12:02:24.615924 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 12:02:24.615924 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 12:02:24.615924 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 12:02:24.616906 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:02:24.616906 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:02:24.616906 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 12:02:24.616906 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 12:02:24.616906 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 12:02:24.616906 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 12:02:24.616906 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:02:24.616906 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:02:24.616906 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:02:24.616906 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:02:24.616906 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:02:24.616906 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 25 12:02:24.616906 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:02:24.616906 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:02:24.616906 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:02:24.616906 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:02:24.616906 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:02:24.616906 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:02:24.617850 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:02:24.617850 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:02:24.617850 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 12:02:24.617850 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 12:02:24.617850 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 12:02:24.617850 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 12:02:24.617850 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 12:02:24.617850 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 12:02:24.617850 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 25 12:02:24.617850 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 25 12:02:24.621850 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 401/792] 10.222.0.42 () {56 vars in 1096 bytes} [Mon Jul 25 12:02:24 2022] GET /identity/v3/users/swift => generated 83 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:02:24.623048 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ffde12-9cf8-4789-be0a-694be7112419 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:24.626786 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9592ce2-6a15-42ac-9013-147077ed7fe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:24.627744 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9592ce2-6a15-42ac-9013-147077ed7fe3 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:24.628308 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9592ce2-6a15-42ac-9013-147077ed7fe3 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:24.628807 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9592ce2-6a15-42ac-9013-147077ed7fe3 None admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:24.629780 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9592ce2-6a15-42ac-9013-147077ed7fe3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:02:24.630288 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9592ce2-6a15-42ac-9013-147077ed7fe3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:24.631912 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9592ce2-6a15-42ac-9013-147077ed7fe3 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:24.647953 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 392/793] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jul 25 12:02:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:02:24.652940 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9592ce2-6a15-42ac-9013-147077ed7fe3 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:24.725682 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:24.767978 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ffde12-9cf8-4789-be0a-694be7112419 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:24.800741 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9592ce2-6a15-42ac-9013-147077ed7fe3 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:24.840400 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7506f3-46e7-4260-9e38-482d8912f1a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:24.841228 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7506f3-46e7-4260-9e38-482d8912f1a7 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:24.841515 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7506f3-46e7-4260-9e38-482d8912f1a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:24.841790 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7506f3-46e7-4260-9e38-482d8912f1a7 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:24.850384 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7506f3-46e7-4260-9e38-482d8912f1a7 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:24.851876 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7506f3-46e7-4260-9e38-482d8912f1a7 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:24.857152 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e30363-b43a-4192-a272-8773b789d3ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:24.858093 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e30363-b43a-4192-a272-8773b789d3ed None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:24.858579 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e30363-b43a-4192-a272-8773b789d3ed None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:24.859056 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e30363-b43a-4192-a272-8773b789d3ed None admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:24.860167 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e30363-b43a-4192-a272-8773b789d3ed None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:02:24.861199 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e30363-b43a-4192-a272-8773b789d3ed None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:24.862837 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e30363-b43a-4192-a272-8773b789d3ed None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:24.869559 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-2a7506f3-46e7-4260-9e38-482d8912f1a7 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 25 12:02:24.869559 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:02:24.869559 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:02:24.869559 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:02:24.869559 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:02:24.869559 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:02:24.869559 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:02:24.869559 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:02:24.869559 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:02:24.869559 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:02:24.869559 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:02:24.869559 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:02:24.869559 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 12:02:24.869559 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 12:02:24.869559 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 12:02:24.869559 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 12:02:24.870526 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:02:24.870526 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:02:24.870526 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 12:02:24.870526 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 12:02:24.870526 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 12:02:24.870526 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 12:02:24.870526 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:02:24.870526 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:02:24.870526 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:02:24.870526 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:02:24.870526 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:02:24.870526 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jul 25 12:02:24.870526 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:02:24.870526 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:02:24.870526 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:02:24.870526 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:02:24.870526 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:02:24.870526 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:02:24.871473 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:02:24.871473 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:02:24.871473 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 12:02:24.871473 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 12:02:24.871473 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 12:02:24.871473 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 12:02:24.871473 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 12:02:24.871473 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 12:02:24.871473 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 25 12:02:24.871473 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jul 25 12:02:24.871473 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 393/794] 10.222.0.42 () {56 vars in 1117 bytes} [Mon Jul 25 12:02:24 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:02:24.875046 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7506f3-46e7-4260-9e38-482d8912f1a7 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:24.903350 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 402/795] 10.222.0.42 () {56 vars in 1135 bytes} [Mon Jul 25 12:02:24 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:02:24.908630 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e30363-b43a-4192-a272-8773b789d3ed None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:24.951603 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:24.993115 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7506f3-46e7-4260-9e38-482d8912f1a7 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:25.020667 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:25.034831 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b97b52-63e2-482d-9d5f-fb9d7baa62b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:25.035795 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b97b52-63e2-482d-9d5f-fb9d7baa62b6 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:25.036231 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b97b52-63e2-482d-9d5f-fb9d7baa62b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:25.036678 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b97b52-63e2-482d-9d5f-fb9d7baa62b6 None admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:25.038003 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b97b52-63e2-482d-9d5f-fb9d7baa62b6 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:02:25.038466 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b97b52-63e2-482d-9d5f-fb9d7baa62b6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:25.041525 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b97b52-63e2-482d-9d5f-fb9d7baa62b6 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:25.063021 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 394/796] 10.222.0.42 () {56 vars in 1149 bytes} [Mon Jul 25 12:02:24 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 12:02:25.070115 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b97b52-63e2-482d-9d5f-fb9d7baa62b6 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:25.181764 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e30363-b43a-4192-a272-8773b789d3ed None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:25.186001 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b97b52-63e2-482d-9d5f-fb9d7baa62b6 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:25.225061 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31f97dd-24b2-4364-b17d-7aedc699332b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:25.225061 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31f97dd-24b2-4364-b17d-7aedc699332b None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:25.225061 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31f97dd-24b2-4364-b17d-7aedc699332b None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:25.225424 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31f97dd-24b2-4364-b17d-7aedc699332b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:25.231278 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a238f9-6ffa-4bb8-b8de-5d5d5735c6a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:25.232854 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a238f9-6ffa-4bb8-b8de-5d5d5735c6a5 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:25.233323 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a238f9-6ffa-4bb8-b8de-5d5d5735c6a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:25.233798 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a238f9-6ffa-4bb8-b8de-5d5d5735c6a5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:25.236778 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31f97dd-24b2-4364-b17d-7aedc699332b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:25.238386 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31f97dd-24b2-4364-b17d-7aedc699332b None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:25.248160 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a238f9-6ffa-4bb8-b8de-5d5d5735c6a5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:25.250068 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a238f9-6ffa-4bb8-b8de-5d5d5735c6a5 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:25.256624 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-e31f97dd-24b2-4364-b17d-7aedc699332b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:02:25.256624 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:02:25.256624 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:02:25.256624 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:02:25.256624 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:02:25.256624 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:02:25.256624 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:02:25.256624 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:02:25.256624 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:02:25.256624 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:02:25.256624 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:02:25.256624 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:02:25.256624 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 12:02:25.256624 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 12:02:25.256624 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 12:02:25.257507 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 12:02:25.257507 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:02:25.257507 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:02:25.257507 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:02:25.257507 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:02:25.257507 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:02:25.257507 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:02:25.257507 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:02:25.257507 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jul 25 12:02:25.257507 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:02:25.257507 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:02:25.257507 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:02:25.257507 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:02:25.257507 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:02:25.257507 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:02:25.257507 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:02:25.257507 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:02:25.258383 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 12:02:25.258383 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 12:02:25.258383 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:02:25.258383 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jul 25 12:02:25.262469 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 395/797] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jul 25 12:02:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 12:02:25.263897 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2bf8e8-a820-4a0c-bd57-fdf13cc6bf79 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:25.264194 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2bf8e8-a820-4a0c-bd57-fdf13cc6bf79 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:25.264483 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2bf8e8-a820-4a0c-bd57-fdf13cc6bf79 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:25.266048 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 396/798] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 25 12:02:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:02:25.266797 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2bf8e8-a820-4a0c-bd57-fdf13cc6bf79 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:25.275496 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-89a238f9-6ffa-4bb8-b8de-5d5d5735c6a5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:02:25.275496 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:02:25.275496 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:02:25.275496 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:02:25.275496 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:02:25.275496 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:02:25.275496 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:02:25.275496 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:02:25.275496 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:02:25.275496 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:02:25.275496 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:02:25.275496 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:02:25.275496 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 12:02:25.275496 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 12:02:25.275496 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 12:02:25.276504 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 12:02:25.276504 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:02:25.276504 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:02:25.276504 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:02:25.276504 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:02:25.276504 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:02:25.276504 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:02:25.276504 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:02:25.276504 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 25 12:02:25.276504 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:02:25.276504 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:02:25.276504 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:02:25.276504 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:02:25.276504 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:02:25.276504 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:02:25.276504 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:02:25.276504 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:02:25.277390 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 12:02:25.277390 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 12:02:25.277390 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:02:25.277390 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 25 12:02:25.280136 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 403/799] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jul 25 12:02:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 372 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 12:02:25.282094 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d261493-e596-4771-9e6e-a864908624b6 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:25.283133 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d261493-e596-4771-9e6e-a864908624b6 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:25.283667 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d261493-e596-4771-9e6e-a864908624b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:25.308173 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-0d261493-e596-4771-9e6e-a864908624b6 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:02:25.309474 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-0d261493-e596-4771-9e6e-a864908624b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:02:25.315300 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:25.387872 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 404/800] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 25 12:02:25 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 12:02:25.389134 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d261493-e596-4771-9e6e-a864908624b6 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:25.392532 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:25.404071 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2bf8e8-a820-4a0c-bd57-fdf13cc6bf79 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:25.442323 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0752f799-bf02-4714-b5f4-a9e9ceb8ac3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:25.442993 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0752f799-bf02-4714-b5f4-a9e9ceb8ac3c None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:25.443280 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0752f799-bf02-4714-b5f4-a9e9ceb8ac3c None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:25.443584 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0752f799-bf02-4714-b5f4-a9e9ceb8ac3c None admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:25.444314 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0752f799-bf02-4714-b5f4-a9e9ceb8ac3c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:02:25.444632 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0752f799-bf02-4714-b5f4-a9e9ceb8ac3c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:25.445998 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0752f799-bf02-4714-b5f4-a9e9ceb8ac3c None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:25.457386 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 397/801] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jul 25 12:02:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:02:25.458479 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a32792d-e0a2-4501-956a-f4a51969c5fe None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:25.458775 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a32792d-e0a2-4501-956a-f4a51969c5fe None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:25.459059 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a32792d-e0a2-4501-956a-f4a51969c5fe None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:25.463585 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 398/802] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 25 12:02:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:02:25.463585 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a32792d-e0a2-4501-956a-f4a51969c5fe None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:25.531683 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d261493-e596-4771-9e6e-a864908624b6 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:25.572650 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13c6b0f-d79f-45a5-8be1-19204bc0ec2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:25.573329 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13c6b0f-d79f-45a5-8be1-19204bc0ec2b None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:25.573617 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13c6b0f-d79f-45a5-8be1-19204bc0ec2b None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:25.573900 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13c6b0f-d79f-45a5-8be1-19204bc0ec2b None admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:25.574629 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13c6b0f-d79f-45a5-8be1-19204bc0ec2b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:02:25.574941 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13c6b0f-d79f-45a5-8be1-19204bc0ec2b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:25.576325 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13c6b0f-d79f-45a5-8be1-19204bc0ec2b None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:25.576898 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:25.586421 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a32792d-e0a2-4501-956a-f4a51969c5fe None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:25.595985 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 405/803] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jul 25 12:02:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:02:25.596549 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13c6b0f-d79f-45a5-8be1-19204bc0ec2b None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:25.625087 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c27a40-8329-43c1-8efe-7d5ea1e3ccf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:25.625900 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c27a40-8329-43c1-8efe-7d5ea1e3ccf4 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:25.626220 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c27a40-8329-43c1-8efe-7d5ea1e3ccf4 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:25.626502 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c27a40-8329-43c1-8efe-7d5ea1e3ccf4 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:25.627168 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c27a40-8329-43c1-8efe-7d5ea1e3ccf4 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:25.628568 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c27a40-8329-43c1-8efe-7d5ea1e3ccf4 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:25.640604 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-d1c27a40-8329-43c1-8efe-7d5ea1e3ccf4 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 25 12:02:25.640604 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:02:25.640604 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 12:02:25.640604 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 12:02:25.640604 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 12:02:25.640604 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 12:02:25.640604 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 12:02:25.640604 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 12:02:25.640604 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 25 12:02:25.640604 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jul 25 12:02:25.640604 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 12:02:25.640604 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jul 25 12:02:25.640604 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:02:25.640604 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:02:25.640604 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:02:25.640604 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:02:25.641534 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:02:25.641534 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:02:25.641534 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:02:25.641534 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:02:25.641534 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:02:25.641534 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:02:25.641534 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:02:25.641534 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 12:02:25.641534 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 12:02:25.641534 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 12:02:25.641534 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 12:02:25.641534 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:02:25.641534 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:02:25.641534 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:02:25.641534 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:02:25.641534 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:02:25.648130 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:02:25.648130 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:02:25.648130 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jul 25 12:02:25.648130 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:02:25.648130 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:02:25.648130 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:02:25.648130 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:02:25.648130 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:02:25.648130 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:02:25.648130 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:02:25.648130 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:02:25.648130 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 12:02:25.648130 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 12:02:25.648130 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 25 12:02:25.648130 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jul 25 12:02:25.648130 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 399/804] 10.222.0.42 () {56 vars in 1129 bytes} [Mon Jul 25 12:02:25 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:02:25.648959 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a7e678-2c91-40a0-9e0a-52a1c8d98ca4 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:25.648959 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a7e678-2c91-40a0-9e0a-52a1c8d98ca4 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:25.648959 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a7e678-2c91-40a0-9e0a-52a1c8d98ca4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:25.662163 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:25.666109 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-e0a7e678-2c91-40a0-9e0a-52a1c8d98ca4 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:02:25.667515 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-e0a7e678-2c91-40a0-9e0a-52a1c8d98ca4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:02:25.703266 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13c6b0f-d79f-45a5-8be1-19204bc0ec2b None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:25.747569 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 400/805] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 25 12:02:25 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 12:02:25.747752 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812e4b3c-1fa2-423f-8d63-c5aa902d22ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:25.747886 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a7e678-2c91-40a0-9e0a-52a1c8d98ca4 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:25.747975 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812e4b3c-1fa2-423f-8d63-c5aa902d22ad None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:25.747975 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812e4b3c-1fa2-423f-8d63-c5aa902d22ad None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:25.747975 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812e4b3c-1fa2-423f-8d63-c5aa902d22ad None admin] PATH_INFO: `/v3/projects/service` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:25.755987 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812e4b3c-1fa2-423f-8d63-c5aa902d22ad None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:25.757510 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812e4b3c-1fa2-423f-8d63-c5aa902d22ad None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:25.772568 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-812e4b3c-1fa2-423f-8d63-c5aa902d22ad None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 12:02:25.772568 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:02:25.772568 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:02:25.772568 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:02:25.772568 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:02:25.772568 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:02:25.772568 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:02:25.772568 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:02:25.772568 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:02:25.772568 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:02:25.772568 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:02:25.772568 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:02:25.772568 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 12:02:25.772568 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 12:02:25.772568 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 12:02:25.773669 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 12:02:25.773669 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:02:25.773669 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:02:25.773669 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:02:25.773669 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:02:25.773669 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:02:25.773669 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:02:25.773669 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:02:25.773669 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 25 12:02:25.773669 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:02:25.773669 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:02:25.773669 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:02:25.773669 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:02:25.773669 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:02:25.773669 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:02:25.773669 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:02:25.773669 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:02:25.774787 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 12:02:25.774787 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 12:02:25.774787 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 12:02:25.774787 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 12:02:25.774787 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 12:02:25.774787 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 12:02:25.774787 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 12:02:25.774787 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 25 12:02:25.774787 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 406/806] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Jul 25 12:02:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:02:25.775375 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812e4b3c-1fa2-423f-8d63-c5aa902d22ad None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:25.861622 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:25.875658 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a7e678-2c91-40a0-9e0a-52a1c8d98ca4 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:25.898671 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812e4b3c-1fa2-423f-8d63-c5aa902d22ad None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:25.922083 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:25.922354 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bceb27-38ed-4a04-a661-71a389a15bc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:25.924011 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bceb27-38ed-4a04-a661-71a389a15bc2 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:25.924385 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bceb27-38ed-4a04-a661-71a389a15bc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:25.924872 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bceb27-38ed-4a04-a661-71a389a15bc2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:25.933821 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13bceb27-38ed-4a04-a661-71a389a15bc2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:25.935186 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1784de-22b8-4312-82bb-c8c4694108e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:25.935305 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13bceb27-38ed-4a04-a661-71a389a15bc2 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:25.935889 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1784de-22b8-4312-82bb-c8c4694108e8 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:25.936180 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1784de-22b8-4312-82bb-c8c4694108e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:25.936459 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1784de-22b8-4312-82bb-c8c4694108e8 None admin] PATH_INFO: `/v3/services` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:25.937173 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1784de-22b8-4312-82bb-c8c4694108e8 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:02:25.937486 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1784de-22b8-4312-82bb-c8c4694108e8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:25.938872 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1784de-22b8-4312-82bb-c8c4694108e8 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:25.947856 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 407/807] 10.222.0.42 () {56 vars in 1124 bytes} [Mon Jul 25 12:02:25 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:02:25.947856 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1784de-22b8-4312-82bb-c8c4694108e8 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:25.949768 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-13bceb27-38ed-4a04-a661-71a389a15bc2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 12:02:25.949768 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:02:25.949768 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:02:25.949768 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:02:25.949768 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:02:25.949768 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:02:25.949768 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:02:25.949768 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:02:25.949768 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:02:25.949768 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:02:25.949768 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:02:25.949768 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:02:25.949768 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 12:02:25.949768 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 12:02:25.949768 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 12:02:25.950627 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 12:02:25.950627 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:02:25.950627 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:02:25.950627 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:02:25.950627 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:02:25.950627 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:02:25.950627 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:02:25.950627 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:02:25.950627 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jul 25 12:02:25.950627 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:02:25.950627 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:02:25.950627 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:02:25.950627 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:02:25.950627 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:02:25.950627 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:02:25.950627 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:02:25.950627 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:02:25.951528 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 12:02:25.951528 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 12:02:25.951528 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 12:02:25.951528 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 12:02:25.951528 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 12:02:25.951528 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 12:02:25.951528 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 12:02:25.951528 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jul 25 12:02:25.955839 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 401/808] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Jul 25 12:02:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:02:25.956973 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bceb27-38ed-4a04-a661-71a389a15bc2 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:26.058814 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1784de-22b8-4312-82bb-c8c4694108e8 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:26.062990 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bceb27-38ed-4a04-a661-71a389a15bc2 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:26.096743 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf182534-e9bf-497c-a96e-d09f3bfa632a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:26.097472 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf182534-e9bf-497c-a96e-d09f3bfa632a None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:26.097997 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf182534-e9bf-497c-a96e-d09f3bfa632a None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:26.097997 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf182534-e9bf-497c-a96e-d09f3bfa632a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:26.102917 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c35b9d5-7b6d-4ecb-a8d0-2b93b571a1f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:26.103620 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c35b9d5-7b6d-4ecb-a8d0-2b93b571a1f6 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:26.103917 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c35b9d5-7b6d-4ecb-a8d0-2b93b571a1f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:26.104201 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c35b9d5-7b6d-4ecb-a8d0-2b93b571a1f6 None admin] PATH_INFO: `/v3/projects` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:26.104990 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c35b9d5-7b6d-4ecb-a8d0-2b93b571a1f6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:02:26.105307 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c35b9d5-7b6d-4ecb-a8d0-2b93b571a1f6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:26.106735 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c35b9d5-7b6d-4ecb-a8d0-2b93b571a1f6 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:26.108393 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf182534-e9bf-497c-a96e-d09f3bfa632a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 12:02:26.116719 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-cf182534-e9bf-497c-a96e-d09f3bfa632a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 12:02:26.116719 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:02:26.116719 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:02:26.116719 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:02:26.116719 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:02:26.116719 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:02:26.116719 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:02:26.116719 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:02:26.116719 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:02:26.116719 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:02:26.116719 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:02:26.116719 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:02:26.116719 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 12:02:26.116719 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 12:02:26.116719 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 12:02:26.116719 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 12:02:26.117795 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 12:02:26.117795 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 12:02:26.117795 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 12:02:26.117795 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 12:02:26.117795 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:02:26.117795 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:02:26.117795 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:02:26.117795 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:02:26.117795 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:02:26.117795 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:02:26.117795 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:02:26.117795 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 25 12:02:26.117795 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:02:26.117795 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:02:26.117795 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:02:26.117795 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:02:26.118626 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:02:26.118626 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:02:26.118626 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:02:26.118626 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:02:26.118626 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 12:02:26.118626 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 12:02:26.118626 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 12:02:26.118626 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 12:02:26.118626 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 12:02:26.118626 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 12:02:26.118626 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 12:02:26.118626 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 25 12:02:26.118626 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 408/809] 10.222.0.42 () {56 vars in 1096 bytes} [Mon Jul 25 12:02:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:02:26.118626 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf182534-e9bf-497c-a96e-d09f3bfa632a None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:26.125458 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 402/810] 10.222.0.42 () {56 vars in 1148 bytes} [Mon Jul 25 12:02:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:02:26.126339 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c35b9d5-7b6d-4ecb-a8d0-2b93b571a1f6 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:26.139028 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:26.171124 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:26.228714 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf182534-e9bf-497c-a96e-d09f3bfa632a None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:26.238849 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c35b9d5-7b6d-4ecb-a8d0-2b93b571a1f6 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:26.263637 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf415e5-4deb-4f08-8fa8-1db5fc1d5dde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:26.264304 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf415e5-4deb-4f08-8fa8-1db5fc1d5dde None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:26.264589 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf415e5-4deb-4f08-8fa8-1db5fc1d5dde None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:26.264868 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf415e5-4deb-4f08-8fa8-1db5fc1d5dde None admin] PATH_INFO: `/v3/services` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:26.265570 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf415e5-4deb-4f08-8fa8-1db5fc1d5dde None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:02:26.265883 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf415e5-4deb-4f08-8fa8-1db5fc1d5dde None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:26.267787 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf415e5-4deb-4f08-8fa8-1db5fc1d5dde None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:26.272856 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 409/811] 10.222.0.42 () {56 vars in 1124 bytes} [Mon Jul 25 12:02:26 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:02:26.273628 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf415e5-4deb-4f08-8fa8-1db5fc1d5dde None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:26.278075 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c8a579-6403-4d47-951a-dbc956c1ee89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:26.278949 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c8a579-6403-4d47-951a-dbc956c1ee89 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:26.279429 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c8a579-6403-4d47-951a-dbc956c1ee89 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:26.279743 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c8a579-6403-4d47-951a-dbc956c1ee89 None admin] PATH_INFO: `/v3/projects` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:26.280541 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c8a579-6403-4d47-951a-dbc956c1ee89 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:02:26.280861 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c8a579-6403-4d47-951a-dbc956c1ee89 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:26.282284 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c8a579-6403-4d47-951a-dbc956c1ee89 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:26.301125 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 403/812] 10.222.0.42 () {56 vars in 1148 bytes} [Mon Jul 25 12:02:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:02:26.301969 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c8a579-6403-4d47-951a-dbc956c1ee89 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:26.372638 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:26.384412 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf415e5-4deb-4f08-8fa8-1db5fc1d5dde None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:26.397602 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:26.410168 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c8a579-6403-4d47-951a-dbc956c1ee89 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:26.424899 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7a2ebd-3fa3-4b89-a489-e6cb309a5a85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:26.425571 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7a2ebd-3fa3-4b89-a489-e6cb309a5a85 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:26.425854 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7a2ebd-3fa3-4b89-a489-e6cb309a5a85 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:26.426129 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7a2ebd-3fa3-4b89-a489-e6cb309a5a85 None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:26.426896 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7a2ebd-3fa3-4b89-a489-e6cb309a5a85 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:02:26.427202 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7a2ebd-3fa3-4b89-a489-e6cb309a5a85 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:26.428563 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7a2ebd-3fa3-4b89-a489-e6cb309a5a85 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:26.437426 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 410/813] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Jul 25 12:02:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:02:26.438465 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7a2ebd-3fa3-4b89-a489-e6cb309a5a85 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:26.448009 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1106b5-933a-433a-b0ff-030dc4975752 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:26.449131 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1106b5-933a-433a-b0ff-030dc4975752 None admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:26.449624 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1106b5-933a-433a-b0ff-030dc4975752 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:26.450104 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1106b5-933a-433a-b0ff-030dc4975752 None admin] PATH_INFO: `/v3/projects/b9f62b8f37304b8a9a4bb54205347577/users/93207b858d294d049fa56d45810a40aa/roles/79dec36aee5b4322bf652041c25ca6e4` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:26.466803 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1106b5-933a-433a-b0ff-030dc4975752 None admin] RBAC: Authorizing `identity:create_grant(project_id=b9f62b8f37304b8a9a4bb54205347577, user_id=93207b858d294d049fa56d45810a40aa, role_id=79dec36aee5b4322bf652041c25ca6e4)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:26.468541 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1106b5-933a-433a-b0ff-030dc4975752 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:26.483627 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 404/814] 10.222.0.42 () {58 vars in 1439 bytes} [Mon Jul 25 12:02:26 2022] PUT /identity/v3/projects/b9f62b8f37304b8a9a4bb54205347577/users/93207b858d294d049fa56d45810a40aa/roles/79dec36aee5b4322bf652041c25ca6e4 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:02:26.484749 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1106b5-933a-433a-b0ff-030dc4975752 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:26.567801 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7a2ebd-3fa3-4b89-a489-e6cb309a5a85 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:26.599870 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1106b5-933a-433a-b0ff-030dc4975752 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:26.619575 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ba266c-5935-4819-b174-99e755df1bd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:26.620384 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ba266c-5935-4819-b174-99e755df1bd3 None admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:26.620740 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ba266c-5935-4819-b174-99e755df1bd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:26.621110 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ba266c-5935-4819-b174-99e755df1bd3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:26.621946 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ba266c-5935-4819-b174-99e755df1bd3 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:26.623479 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ba266c-5935-4819-b174-99e755df1bd3 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:26.624753 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:26.663786 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 411/815] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Jul 25 12:02:26 2022] POST /identity/v3/endpoints => generated 346 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:02:26.665534 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a900b37-5326-449d-8c0c-84c44616c859 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:26.666103 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ba266c-5935-4819-b174-99e755df1bd3 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:26.667463 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a900b37-5326-449d-8c0c-84c44616c859 None admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:26.667791 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a900b37-5326-449d-8c0c-84c44616c859 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:26.668546 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a900b37-5326-449d-8c0c-84c44616c859 None admin] PATH_INFO: `/v3/projects/b9f62b8f37304b8a9a4bb54205347577/users/7bd782ea66364eeb96592096eae63d4f/roles/a7d20885b6d94928a34b3cc5b3a00006` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:26.673736 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a900b37-5326-449d-8c0c-84c44616c859 None admin] RBAC: Authorizing `identity:create_grant(project_id=b9f62b8f37304b8a9a4bb54205347577, user_id=7bd782ea66364eeb96592096eae63d4f, role_id=a7d20885b6d94928a34b3cc5b3a00006)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:26.675244 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a900b37-5326-449d-8c0c-84c44616c859 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:26.701666 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 405/816] 10.222.0.42 () {58 vars in 1439 bytes} [Mon Jul 25 12:02:26 2022] PUT /identity/v3/projects/b9f62b8f37304b8a9a4bb54205347577/users/7bd782ea66364eeb96592096eae63d4f/roles/a7d20885b6d94928a34b3cc5b3a00006 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:02:26.718481 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:26.816300 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ba266c-5935-4819-b174-99e755df1bd3 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:26.837530 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5b75ac-3caf-48c4-b042-57c3803e204e None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:26.837854 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5b75ac-3caf-48c4-b042-57c3803e204e None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:26.838103 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5b75ac-3caf-48c4-b042-57c3803e204e None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:26.841108 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 406/817] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 25 12:02:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 12:02:26.846259 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706d989a-dd28-47bd-9e2f-711fa79f0649 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:26.846539 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706d989a-dd28-47bd-9e2f-711fa79f0649 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:26.846826 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706d989a-dd28-47bd-9e2f-711fa79f0649 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:26.857506 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45ba38c-7dc8-471c-a17b-eab273d3d11d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:26.858334 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45ba38c-7dc8-471c-a17b-eab273d3d11d None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:26.858626 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45ba38c-7dc8-471c-a17b-eab273d3d11d None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:26.858911 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45ba38c-7dc8-471c-a17b-eab273d3d11d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:26.866876 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-706d989a-dd28-47bd-9e2f-711fa79f0649 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:02:26.868156 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-706d989a-dd28-47bd-9e2f-711fa79f0649 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:02:26.875348 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45ba38c-7dc8-471c-a17b-eab273d3d11d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:26.878157 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45ba38c-7dc8-471c-a17b-eab273d3d11d None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:26.903133 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-d45ba38c-7dc8-471c-a17b-eab273d3d11d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:02:26.903133 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:02:26.903133 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:02:26.903133 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:02:26.903133 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:02:26.903133 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:02:26.903133 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:02:26.903133 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:02:26.903133 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:02:26.903133 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:02:26.903133 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:02:26.903133 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:02:26.903133 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 12:02:26.903133 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 12:02:26.903133 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 12:02:26.904192 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 12:02:26.904192 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:02:26.904192 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:02:26.904192 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:02:26.904192 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:02:26.904192 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:02:26.904192 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:02:26.904192 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:02:26.904192 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 25 12:02:26.904192 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:02:26.904192 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:02:26.904192 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:02:26.904192 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:02:26.904192 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:02:26.904192 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:02:26.904192 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:02:26.904192 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:02:26.905076 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 12:02:26.905076 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 12:02:26.905076 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:02:26.905076 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 25 12:02:26.905076 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 412/818] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jul 25 12:02:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:02:26.912062 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45ba38c-7dc8-471c-a17b-eab273d3d11d None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:26.960619 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 407/819] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 25 12:02:26 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 25 12:02:26.981145 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706d989a-dd28-47bd-9e2f-711fa79f0649 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:27.021055 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:27.036259 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45ba38c-7dc8-471c-a17b-eab273d3d11d None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:27.075999 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1dc142-b02e-4764-b136-cbb4fa4988e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:27.076817 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1dc142-b02e-4764-b136-cbb4fa4988e1 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:27.077103 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1dc142-b02e-4764-b136-cbb4fa4988e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:27.077638 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1dc142-b02e-4764-b136-cbb4fa4988e1 None admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:27.078394 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1dc142-b02e-4764-b136-cbb4fa4988e1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:02:27.078704 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1dc142-b02e-4764-b136-cbb4fa4988e1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:27.080151 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1dc142-b02e-4764-b136-cbb4fa4988e1 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:27.095111 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706d989a-dd28-47bd-9e2f-711fa79f0649 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:27.096041 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 413/820] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jul 25 12:02:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:02:27.102052 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1dc142-b02e-4764-b136-cbb4fa4988e1 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:27.109293 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:27.138889 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e52e070-8c69-4bd5-8b7c-3c24d6d74123 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:27.140487 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e52e070-8c69-4bd5-8b7c-3c24d6d74123 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:27.140780 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e52e070-8c69-4bd5-8b7c-3c24d6d74123 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:27.141062 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e52e070-8c69-4bd5-8b7c-3c24d6d74123 None admin] PATH_INFO: `/v3/services/network` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:27.141772 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e52e070-8c69-4bd5-8b7c-3c24d6d74123 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:27.143162 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e52e070-8c69-4bd5-8b7c-3c24d6d74123 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:27.158642 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-2e52e070-8c69-4bd5-8b7c-3c24d6d74123 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 25 12:02:27.158642 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:02:27.158642 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 12:02:27.158642 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 12:02:27.158642 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 12:02:27.158642 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 12:02:27.158642 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 12:02:27.158642 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 12:02:27.158642 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 25 12:02:27.158642 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jul 25 12:02:27.158642 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 12:02:27.158642 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jul 25 12:02:27.158642 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:02:27.158642 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:02:27.158642 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:02:27.158642 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:02:27.158642 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:02:27.159798 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:02:27.159798 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:02:27.159798 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:02:27.159798 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:02:27.159798 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:02:27.159798 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:02:27.159798 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 12:02:27.159798 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 12:02:27.159798 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 12:02:27.159798 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 12:02:27.159798 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:02:27.159798 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:02:27.159798 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:02:27.159798 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:02:27.159798 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:02:27.159798 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:02:27.160642 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:02:27.160642 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jul 25 12:02:27.160642 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:02:27.160642 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:02:27.160642 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:02:27.160642 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:02:27.160642 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:02:27.160642 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:02:27.160642 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:02:27.160642 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:02:27.160642 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 12:02:27.160642 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 12:02:27.160642 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 25 12:02:27.160642 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jul 25 12:02:27.163631 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 408/821] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Jul 25 12:02:26 2022] GET /identity/v3/services/network => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:02:27.168193 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e52e070-8c69-4bd5-8b7c-3c24d6d74123 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:27.223446 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1dc142-b02e-4764-b136-cbb4fa4988e1 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:27.266460 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6b40e9-1222-4578-a9cd-f46e00f183b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:27.268066 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6b40e9-1222-4578-a9cd-f46e00f183b6 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:27.268515 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6b40e9-1222-4578-a9cd-f46e00f183b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:27.268938 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6b40e9-1222-4578-a9cd-f46e00f183b6 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:27.273703 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:27.281889 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6b40e9-1222-4578-a9cd-f46e00f183b6 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:27.284322 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6b40e9-1222-4578-a9cd-f46e00f183b6 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:27.296389 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e52e070-8c69-4bd5-8b7c-3c24d6d74123 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:27.305451 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-bf6b40e9-1222-4578-a9cd-f46e00f183b6 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 25 12:02:27.305451 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:02:27.305451 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:02:27.305451 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:02:27.305451 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:02:27.305451 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:02:27.305451 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:02:27.305451 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:02:27.305451 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:02:27.305451 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:02:27.305451 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:02:27.305451 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:02:27.305451 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 12:02:27.305451 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 12:02:27.305451 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 12:02:27.305451 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 12:02:27.306385 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:02:27.306385 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:02:27.306385 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 12:02:27.306385 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 12:02:27.306385 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 12:02:27.306385 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 12:02:27.306385 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:02:27.306385 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:02:27.306385 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:02:27.306385 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:02:27.306385 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:02:27.306385 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 25 12:02:27.306385 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:02:27.306385 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:02:27.306385 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:02:27.306385 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:02:27.306385 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:02:27.306385 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:02:27.307345 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:02:27.307345 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:02:27.307345 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 12:02:27.307345 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 12:02:27.307345 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 12:02:27.307345 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 12:02:27.307345 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 12:02:27.307345 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 12:02:27.307345 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 25 12:02:27.307345 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 25 12:02:27.309394 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 414/822] 10.222.0.42 () {56 vars in 1093 bytes} [Mon Jul 25 12:02:27 2022] GET /identity/v3/users/nova => generated 82 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:02:27.316498 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6b40e9-1222-4578-a9cd-f46e00f183b6 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:27.347415 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ed5e2f-1f14-4a4b-9a63-c47dc3fd231c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:27.348520 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ed5e2f-1f14-4a4b-9a63-c47dc3fd231c None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:27.348809 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ed5e2f-1f14-4a4b-9a63-c47dc3fd231c None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:27.349090 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ed5e2f-1f14-4a4b-9a63-c47dc3fd231c None admin] PATH_INFO: `/v3/services` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:27.349808 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ed5e2f-1f14-4a4b-9a63-c47dc3fd231c None admin] RBAC: Adding query filter params (name=network) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:02:27.350118 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ed5e2f-1f14-4a4b-9a63-c47dc3fd231c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:27.354577 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ed5e2f-1f14-4a4b-9a63-c47dc3fd231c None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:27.365714 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 409/823] 10.222.0.42 () {56 vars in 1112 bytes} [Mon Jul 25 12:02:27 2022] GET /identity/v3/services?name=network => generated 124 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:02:27.372502 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ed5e2f-1f14-4a4b-9a63-c47dc3fd231c None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:27.451106 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:27.472552 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6b40e9-1222-4578-a9cd-f46e00f183b6 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:27.536518 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ed5e2f-1f14-4a4b-9a63-c47dc3fd231c None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:27.544893 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554a4cd4-b8f5-4f10-a910-81bf2137eac2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:27.545595 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554a4cd4-b8f5-4f10-a910-81bf2137eac2 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:27.545887 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554a4cd4-b8f5-4f10-a910-81bf2137eac2 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:27.546168 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554a4cd4-b8f5-4f10-a910-81bf2137eac2 None admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:27.547028 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554a4cd4-b8f5-4f10-a910-81bf2137eac2 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:02:27.547343 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554a4cd4-b8f5-4f10-a910-81bf2137eac2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:27.549380 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554a4cd4-b8f5-4f10-a910-81bf2137eac2 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:27.573823 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7709c0a1-aad3-4f30-80e0-4a90ee799dca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:27.574493 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7709c0a1-aad3-4f30-80e0-4a90ee799dca None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:27.574783 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7709c0a1-aad3-4f30-80e0-4a90ee799dca None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:27.575067 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7709c0a1-aad3-4f30-80e0-4a90ee799dca None admin] PATH_INFO: `/v3/services` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:27.575170 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 415/824] 10.222.0.42 () {56 vars in 1133 bytes} [Mon Jul 25 12:02:27 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:02:27.579864 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7709c0a1-aad3-4f30-80e0-4a90ee799dca None admin] RBAC: Adding query filter params (type=network) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:02:27.580194 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7709c0a1-aad3-4f30-80e0-4a90ee799dca None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:27.581625 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7709c0a1-aad3-4f30-80e0-4a90ee799dca None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:27.587164 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 410/825] 10.222.0.42 () {56 vars in 1112 bytes} [Mon Jul 25 12:02:27 2022] GET /identity/v3/services?type=network => generated 350 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:02:27.588754 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554a4cd4-b8f5-4f10-a910-81bf2137eac2 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:27.593206 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7709c0a1-aad3-4f30-80e0-4a90ee799dca None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:27.695406 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:27.756808 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554a4cd4-b8f5-4f10-a910-81bf2137eac2 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:27.792536 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:27.796337 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7709c0a1-aad3-4f30-80e0-4a90ee799dca None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:27.837197 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a595372-00f8-4d4b-b29d-cc5bd3ba1c41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:27.838131 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a595372-00f8-4d4b-b29d-cc5bd3ba1c41 None admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:27.838521 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a595372-00f8-4d4b-b29d-cc5bd3ba1c41 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:27.838932 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a595372-00f8-4d4b-b29d-cc5bd3ba1c41 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:27.839897 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a595372-00f8-4d4b-b29d-cc5bd3ba1c41 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:27.841345 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a595372-00f8-4d4b-b29d-cc5bd3ba1c41 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:27.870054 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 411/826] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Jul 25 12:02:27 2022] POST /identity/v3/endpoints => generated 327 bytes in 277 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 12:02:27.887642 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9de3e4-3952-46e8-b867-aded6923c3a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:27.888317 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9de3e4-3952-46e8-b867-aded6923c3a0 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:27.888676 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9de3e4-3952-46e8-b867-aded6923c3a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:27.888948 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9de3e4-3952-46e8-b867-aded6923c3a0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:27.976356 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9de3e4-3952-46e8-b867-aded6923c3a0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:27.977994 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9de3e4-3952-46e8-b867-aded6923c3a0 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:28.053410 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:28.127149 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 416/827] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Jul 25 12:02:27 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 539 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 25 12:02:28.134021 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a595372-00f8-4d4b-b29d-cc5bd3ba1c41 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:28.239421 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a595372-00f8-4d4b-b29d-cc5bd3ba1c41 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:28.277047 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef19d31d-3b79-4122-a559-1e3bf8c6a28a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:28.277852 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef19d31d-3b79-4122-a559-1e3bf8c6a28a None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:28.278139 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef19d31d-3b79-4122-a559-1e3bf8c6a28a None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:28.278422 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef19d31d-3b79-4122-a559-1e3bf8c6a28a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:28.289484 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef19d31d-3b79-4122-a559-1e3bf8c6a28a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:28.291267 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef19d31d-3b79-4122-a559-1e3bf8c6a28a None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:28.308645 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-ef19d31d-3b79-4122-a559-1e3bf8c6a28a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:02:28.308645 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:02:28.308645 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:02:28.308645 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:02:28.308645 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:02:28.308645 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:02:28.308645 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:02:28.308645 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:02:28.308645 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:02:28.308645 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:02:28.308645 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:02:28.308645 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:02:28.308645 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 12:02:28.308645 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 12:02:28.308645 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 12:02:28.309525 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 12:02:28.309525 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:02:28.309525 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:02:28.309525 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:02:28.309525 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:02:28.309525 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:02:28.309525 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:02:28.309525 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:02:28.309525 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jul 25 12:02:28.309525 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:02:28.309525 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:02:28.309525 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:02:28.309525 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:02:28.309525 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:02:28.309525 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:02:28.309525 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:02:28.309525 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:02:28.310409 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 12:02:28.310409 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 12:02:28.310409 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:02:28.310409 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jul 25 12:02:28.310409 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 412/828] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jul 25 12:02:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:02:28.320631 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9de3e4-3952-46e8-b867-aded6923c3a0 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:28.436610 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9de3e4-3952-46e8-b867-aded6923c3a0 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:28.477590 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e3f155-a443-4eb7-8ffa-14d28b2ee9c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:28.478553 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e3f155-a443-4eb7-8ffa-14d28b2ee9c4 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:28.479022 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e3f155-a443-4eb7-8ffa-14d28b2ee9c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:28.479510 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e3f155-a443-4eb7-8ffa-14d28b2ee9c4 None admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:28.480501 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e3f155-a443-4eb7-8ffa-14d28b2ee9c4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:02:28.480990 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e3f155-a443-4eb7-8ffa-14d28b2ee9c4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:28.482661 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e3f155-a443-4eb7-8ffa-14d28b2ee9c4 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:28.498782 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 417/829] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jul 25 12:02:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:02:28.505159 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef19d31d-3b79-4122-a559-1e3bf8c6a28a None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:28.517869 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:28.615788 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef19d31d-3b79-4122-a559-1e3bf8c6a28a None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:28.651768 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d12647-e694-4923-aa73-e8c91bc9facf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:28.652891 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d12647-e694-4923-aa73-e8c91bc9facf None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:28.653373 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d12647-e694-4923-aa73-e8c91bc9facf None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:28.653843 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d12647-e694-4923-aa73-e8c91bc9facf None admin] PATH_INFO: `/v3/projects/service` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:28.661856 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d12647-e694-4923-aa73-e8c91bc9facf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:28.663501 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d12647-e694-4923-aa73-e8c91bc9facf None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:28.678484 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-f2d12647-e694-4923-aa73-e8c91bc9facf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 12:02:28.678484 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:02:28.678484 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:02:28.678484 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:02:28.678484 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:02:28.678484 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:02:28.678484 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:02:28.678484 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:02:28.678484 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:02:28.678484 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:02:28.678484 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:02:28.678484 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:02:28.678484 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 12:02:28.678484 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 12:02:28.678484 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 12:02:28.679422 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 12:02:28.679422 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:02:28.679422 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:02:28.679422 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:02:28.679422 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:02:28.679422 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:02:28.679422 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:02:28.679422 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:02:28.679422 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jul 25 12:02:28.679422 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:02:28.679422 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:02:28.679422 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:02:28.679422 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:02:28.679422 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:02:28.679422 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:02:28.679422 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:02:28.679422 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:02:28.680539 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 12:02:28.680539 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 12:02:28.680539 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 12:02:28.680539 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 12:02:28.680539 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 12:02:28.680539 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 12:02:28.680539 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 12:02:28.680539 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jul 25 12:02:28.682615 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 413/830] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Jul 25 12:02:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:02:28.688503 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e3f155-a443-4eb7-8ffa-14d28b2ee9c4 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:28.701764 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:28.808341 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e3f155-a443-4eb7-8ffa-14d28b2ee9c4 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:28.844152 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fd5e69-b5ab-4dc0-9808-90aa8c74500e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:28.844850 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fd5e69-b5ab-4dc0-9808-90aa8c74500e None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:28.845141 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fd5e69-b5ab-4dc0-9808-90aa8c74500e None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:28.845424 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fd5e69-b5ab-4dc0-9808-90aa8c74500e None admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:28.846278 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fd5e69-b5ab-4dc0-9808-90aa8c74500e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:02:28.846598 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fd5e69-b5ab-4dc0-9808-90aa8c74500e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:28.848057 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fd5e69-b5ab-4dc0-9808-90aa8c74500e None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:28.861491 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 418/831] 10.222.0.42 () {56 vars in 1148 bytes} [Mon Jul 25 12:02:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:02:28.867493 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d12647-e694-4923-aa73-e8c91bc9facf None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:28.939042 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:28.974432 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d12647-e694-4923-aa73-e8c91bc9facf None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:29.010531 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc40956e-9ee9-4e41-bc78-97578b20665c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:29.010766 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254bfde7-afbb-46f3-a55c-8b0693f38bb5 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:29.011018 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254bfde7-afbb-46f3-a55c-8b0693f38bb5 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:29.011184 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc40956e-9ee9-4e41-bc78-97578b20665c None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:29.011279 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254bfde7-afbb-46f3-a55c-8b0693f38bb5 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:29.011476 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc40956e-9ee9-4e41-bc78-97578b20665c None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:29.011793 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc40956e-9ee9-4e41-bc78-97578b20665c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:29.012637 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc40956e-9ee9-4e41-bc78-97578b20665c None admin] RBAC: Adding query filter params (role.id=a7d20885b6d94928a34b3cc5b3a00006, scope.project.id=b9f62b8f37304b8a9a4bb54205347577, user.id=f67142a1193d47c7bd33d7040268233c) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:02:29.012929 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 419/832] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 25 12:02:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 12:02:29.013040 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc40956e-9ee9-4e41-bc78-97578b20665c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:29.014519 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc40956e-9ee9-4e41-bc78-97578b20665c None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:29.018104 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70abf51d-5031-4d11-8323-93514baa883a None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:29.018402 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70abf51d-5031-4d11-8323-93514baa883a None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:29.018692 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70abf51d-5031-4d11-8323-93514baa883a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:29.021631 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 414/833] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Jul 25 12:02:28 2022] GET /identity/v3/role_assignments?user.id=f67142a1193d47c7bd33d7040268233c&scope.project.id=b9f62b8f37304b8a9a4bb54205347577&role.id=a7d20885b6d94928a34b3cc5b3a00006 => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:02:29.038344 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-70abf51d-5031-4d11-8323-93514baa883a None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:02:29.039640 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-70abf51d-5031-4d11-8323-93514baa883a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:02:29.071102 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b88c1c6-7df9-4720-8675-dcdaa841be93 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:29.071571 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b88c1c6-7df9-4720-8675-dcdaa841be93 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:29.071959 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b88c1c6-7df9-4720-8675-dcdaa841be93 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:29.073752 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 415/834] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 25 12:02:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 12:02:29.079014 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ba611a-27e0-4a7b-a342-91e53cfbb5bb None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:29.079456 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ba611a-27e0-4a7b-a342-91e53cfbb5bb None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:29.079954 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ba611a-27e0-4a7b-a342-91e53cfbb5bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:29.101492 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-27ba611a-27e0-4a7b-a342-91e53cfbb5bb None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:02:29.102863 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-27ba611a-27e0-4a7b-a342-91e53cfbb5bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:02:29.124494 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:29.133117 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 420/835] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 25 12:02:29 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 12:02:29.156816 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70abf51d-5031-4d11-8323-93514baa883a None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:29.187659 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 416/836] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 25 12:02:29 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 12:02:29.210310 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ba611a-27e0-4a7b-a342-91e53cfbb5bb None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:29.272538 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70abf51d-5031-4d11-8323-93514baa883a None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:29.289529 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:29.309486 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b02b1f-3c40-4244-8182-963da346c158 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:29.310288 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b02b1f-3c40-4244-8182-963da346c158 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:29.310575 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b02b1f-3c40-4244-8182-963da346c158 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:29.310857 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b02b1f-3c40-4244-8182-963da346c158 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:29.323024 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b02b1f-3c40-4244-8182-963da346c158 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 12:02:29.327446 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ba611a-27e0-4a7b-a342-91e53cfbb5bb None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:29.331163 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-09b02b1f-3c40-4244-8182-963da346c158 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 12:02:29.331163 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:02:29.331163 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:02:29.331163 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:02:29.331163 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:02:29.331163 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:02:29.331163 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:02:29.331163 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:02:29.331163 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:02:29.331163 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:02:29.331163 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:02:29.331163 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:02:29.331163 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 12:02:29.331163 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 12:02:29.331163 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 12:02:29.331163 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 12:02:29.332163 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 12:02:29.332163 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 12:02:29.332163 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 12:02:29.332163 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 12:02:29.332163 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:02:29.332163 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:02:29.332163 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:02:29.332163 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:02:29.332163 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:02:29.332163 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:02:29.332163 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:02:29.332163 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 25 12:02:29.332163 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:02:29.332163 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:02:29.332163 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:02:29.332163 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:02:29.333599 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:02:29.333599 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:02:29.333599 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:02:29.333599 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:02:29.333599 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 12:02:29.333599 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 12:02:29.333599 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 12:02:29.333599 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 12:02:29.333599 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 12:02:29.333599 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 12:02:29.333599 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 12:02:29.333599 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 25 12:02:29.333599 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 421/837] 10.222.0.42 () {56 vars in 1096 bytes} [Mon Jul 25 12:02:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 12:02:29.334311 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751a61bd-0294-462c-8241-4cd46ce9e0e7 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:29.334311 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751a61bd-0294-462c-8241-4cd46ce9e0e7 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:29.334647 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751a61bd-0294-462c-8241-4cd46ce9e0e7 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:29.336114 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 422/838] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 25 12:02:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:02:29.338092 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751a61bd-0294-462c-8241-4cd46ce9e0e7 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:29.371718 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4996934-73ec-4afd-8673-19a3f20a88ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:29.372617 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4996934-73ec-4afd-8673-19a3f20a88ec None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:29.372909 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4996934-73ec-4afd-8673-19a3f20a88ec None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:29.373187 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4996934-73ec-4afd-8673-19a3f20a88ec None admin] PATH_INFO: `/v3/roles/service` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:29.383883 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4996934-73ec-4afd-8673-19a3f20a88ec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 12:02:29.392058 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-d4996934-73ec-4afd-8673-19a3f20a88ec None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 25 12:02:29.392058 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:02:29.392058 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:02:29.392058 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:02:29.392058 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:02:29.392058 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:02:29.392058 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:02:29.392058 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:02:29.392058 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:02:29.392058 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:02:29.392058 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:02:29.392058 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:02:29.392058 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 12:02:29.392058 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 12:02:29.392058 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 12:02:29.392058 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 12:02:29.393185 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 12:02:29.393185 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 12:02:29.393185 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 12:02:29.393185 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 12:02:29.393185 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:02:29.393185 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:02:29.393185 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:02:29.393185 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:02:29.393185 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:02:29.393185 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:02:29.393185 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:02:29.393185 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jul 25 12:02:29.393185 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:02:29.393185 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:02:29.393185 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:02:29.393185 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:02:29.394384 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:02:29.394384 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:02:29.394384 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:02:29.394384 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:02:29.394384 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 12:02:29.394384 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 12:02:29.394384 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 12:02:29.394384 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 12:02:29.394384 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 12:02:29.394384 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 12:02:29.394384 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 25 12:02:29.394384 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jul 25 12:02:29.394384 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 417/839] 10.222.0.42 () {56 vars in 1102 bytes} [Mon Jul 25 12:02:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:02:29.395276 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8732fb-77b8-4156-b8b7-97474576b6b7 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:29.395276 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8732fb-77b8-4156-b8b7-97474576b6b7 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:29.395431 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8732fb-77b8-4156-b8b7-97474576b6b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:29.411673 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:29.415293 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-4e8732fb-77b8-4156-b8b7-97474576b6b7 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:02:29.416784 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-4e8732fb-77b8-4156-b8b7-97474576b6b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:02:29.457903 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751a61bd-0294-462c-8241-4cd46ce9e0e7 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:29.487589 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 418/840] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 25 12:02:29 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 12:02:29.488466 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8732fb-77b8-4156-b8b7-97474576b6b7 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:29.500677 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622ef943-a995-4c23-88a4-052aba5616c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:29.501309 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622ef943-a995-4c23-88a4-052aba5616c4 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:29.501593 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622ef943-a995-4c23-88a4-052aba5616c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:29.501874 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622ef943-a995-4c23-88a4-052aba5616c4 None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:29.502622 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622ef943-a995-4c23-88a4-052aba5616c4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:02:29.502931 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622ef943-a995-4c23-88a4-052aba5616c4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:29.504406 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622ef943-a995-4c23-88a4-052aba5616c4 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:29.518705 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 423/841] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Jul 25 12:02:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 12:02:29.519624 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622ef943-a995-4c23-88a4-052aba5616c4 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:29.553284 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:29.606407 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8732fb-77b8-4156-b8b7-97474576b6b7 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:29.628274 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622ef943-a995-4c23-88a4-052aba5616c4 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:29.648164 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd632c7-7e8d-4475-a0c5-9b7910a3b321 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:29.649490 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd632c7-7e8d-4475-a0c5-9b7910a3b321 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:29.650217 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd632c7-7e8d-4475-a0c5-9b7910a3b321 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:29.650714 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd632c7-7e8d-4475-a0c5-9b7910a3b321 None admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:29.651201 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:29.651851 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd632c7-7e8d-4475-a0c5-9b7910a3b321 None admin] RBAC: Adding query filter params (name=service) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:02:29.652603 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd632c7-7e8d-4475-a0c5-9b7910a3b321 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:29.654132 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd632c7-7e8d-4475-a0c5-9b7910a3b321 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:29.663923 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb8fb7f-7835-4893-9c5f-b9dbdf3c9d52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:29.664725 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb8fb7f-7835-4893-9c5f-b9dbdf3c9d52 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:29.665013 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb8fb7f-7835-4893-9c5f-b9dbdf3c9d52 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:29.665292 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb8fb7f-7835-4893-9c5f-b9dbdf3c9d52 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:29.666199 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb8fb7f-7835-4893-9c5f-b9dbdf3c9d52 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:29.667572 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb8fb7f-7835-4893-9c5f-b9dbdf3c9d52 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:29.668559 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 419/842] 10.222.0.42 () {56 vars in 1103 bytes} [Mon Jul 25 12:02:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:02:29.669585 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd632c7-7e8d-4475-a0c5-9b7910a3b321 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:29.679425 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-6fb8fb7f-7835-4893-9c5f-b9dbdf3c9d52 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 25 12:02:29.679425 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:02:29.679425 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 12:02:29.679425 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 12:02:29.679425 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 12:02:29.679425 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 12:02:29.679425 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 12:02:29.679425 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 12:02:29.679425 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 25 12:02:29.679425 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 25 12:02:29.679425 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 12:02:29.679425 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 25 12:02:29.679425 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:02:29.679425 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:02:29.679425 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:02:29.679425 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:02:29.679425 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:02:29.685431 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:02:29.685431 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:02:29.685431 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:02:29.685431 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:02:29.685431 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:02:29.685431 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:02:29.685431 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 12:02:29.685431 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 12:02:29.685431 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 12:02:29.685431 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 12:02:29.685431 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:02:29.685431 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:02:29.685431 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:02:29.685431 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:02:29.685431 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:02:29.685431 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:02:29.686350 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:02:29.686350 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 25 12:02:29.686350 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:02:29.686350 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:02:29.686350 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:02:29.686350 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:02:29.686350 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:02:29.686350 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:02:29.686350 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:02:29.686350 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:02:29.686350 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 12:02:29.686350 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 12:02:29.686350 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 25 12:02:29.686350 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 25 12:02:29.686350 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 424/843] 10.222.0.42 () {56 vars in 1114 bytes} [Mon Jul 25 12:02:29 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:02:29.687148 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb8fb7f-7835-4893-9c5f-b9dbdf3c9d52 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:29.786481 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb8fb7f-7835-4893-9c5f-b9dbdf3c9d52 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:29.802018 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd632c7-7e8d-4475-a0c5-9b7910a3b321 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:29.816904 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:29.825897 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cd41b0-7c57-451d-86a2-94ee5f660f28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:29.826782 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cd41b0-7c57-451d-86a2-94ee5f660f28 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:29.827145 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cd41b0-7c57-451d-86a2-94ee5f660f28 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:29.827433 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cd41b0-7c57-451d-86a2-94ee5f660f28 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:29.839102 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03cd41b0-7c57-451d-86a2-94ee5f660f28 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:29.839734 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98edd2e-3fcc-45a3-9747-d692f4118a8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:29.840594 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98edd2e-3fcc-45a3-9747-d692f4118a8a None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:29.840721 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03cd41b0-7c57-451d-86a2-94ee5f660f28 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:29.840971 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98edd2e-3fcc-45a3-9747-d692f4118a8a None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:29.841318 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98edd2e-3fcc-45a3-9747-d692f4118a8a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:29.852694 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98edd2e-3fcc-45a3-9747-d692f4118a8a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:29.854203 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98edd2e-3fcc-45a3-9747-d692f4118a8a None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:29.859070 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-03cd41b0-7c57-451d-86a2-94ee5f660f28 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:02:29.859070 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:02:29.859070 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:02:29.859070 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:02:29.859070 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:02:29.859070 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:02:29.859070 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:02:29.859070 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:02:29.859070 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:02:29.859070 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:02:29.859070 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:02:29.859070 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:02:29.859070 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 12:02:29.859070 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 12:02:29.859070 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 12:02:29.859989 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 12:02:29.859989 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:02:29.859989 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:02:29.859989 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:02:29.859989 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:02:29.859989 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:02:29.859989 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:02:29.859989 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:02:29.859989 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 25 12:02:29.859989 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:02:29.859989 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:02:29.859989 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:02:29.859989 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:02:29.859989 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:02:29.859989 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:02:29.859989 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:02:29.859989 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:02:29.861495 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 12:02:29.861495 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 12:02:29.861495 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:02:29.861495 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 25 12:02:29.861495 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 425/844] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jul 25 12:02:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:02:29.862178 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cd41b0-7c57-451d-86a2-94ee5f660f28 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:29.876444 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-a98edd2e-3fcc-45a3-9747-d692f4118a8a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:02:29.876444 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:02:29.876444 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:02:29.876444 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:02:29.876444 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:02:29.876444 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:02:29.876444 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:02:29.876444 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:02:29.876444 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:02:29.876444 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:02:29.876444 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:02:29.876444 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:02:29.876444 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 12:02:29.876444 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 12:02:29.876444 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 12:02:29.877764 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 12:02:29.877764 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:02:29.877764 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:02:29.877764 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:02:29.877764 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:02:29.877764 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:02:29.877764 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:02:29.877764 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:02:29.877764 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jul 25 12:02:29.877764 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:02:29.877764 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:02:29.877764 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:02:29.877764 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:02:29.877764 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:02:29.877764 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:02:29.877764 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:02:29.877764 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:02:29.890537 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 12:02:29.890537 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 12:02:29.890537 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:02:29.890537 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jul 25 12:02:29.890537 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 420/845] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jul 25 12:02:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:02:29.890537 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98edd2e-3fcc-45a3-9747-d692f4118a8a None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:29.909306 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:29.970218 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cd41b0-7c57-451d-86a2-94ee5f660f28 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:30.003403 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98edd2e-3fcc-45a3-9747-d692f4118a8a None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:30.004975 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f111b3eb-fbb5-440a-8d6f-ee800fcb74a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:30.005672 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f111b3eb-fbb5-440a-8d6f-ee800fcb74a7 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:30.005954 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f111b3eb-fbb5-440a-8d6f-ee800fcb74a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:30.006237 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f111b3eb-fbb5-440a-8d6f-ee800fcb74a7 None admin] PATH_INFO: `/v3/services` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:30.006941 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f111b3eb-fbb5-440a-8d6f-ee800fcb74a7 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:02:30.007249 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f111b3eb-fbb5-440a-8d6f-ee800fcb74a7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:30.008634 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f111b3eb-fbb5-440a-8d6f-ee800fcb74a7 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:30.019073 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 426/846] 10.222.0.42 () {56 vars in 1114 bytes} [Mon Jul 25 12:02:29 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:02:30.019073 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f111b3eb-fbb5-440a-8d6f-ee800fcb74a7 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:30.039276 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb191a89-d3bc-4896-9fb9-d2f18dc242e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:30.040247 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb191a89-d3bc-4896-9fb9-d2f18dc242e4 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:30.040539 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb191a89-d3bc-4896-9fb9-d2f18dc242e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:30.041084 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb191a89-d3bc-4896-9fb9-d2f18dc242e4 None admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:30.041809 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb191a89-d3bc-4896-9fb9-d2f18dc242e4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:02:30.042126 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb191a89-d3bc-4896-9fb9-d2f18dc242e4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:30.043930 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb191a89-d3bc-4896-9fb9-d2f18dc242e4 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:30.064167 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 421/847] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jul 25 12:02:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:02:30.064167 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb191a89-d3bc-4896-9fb9-d2f18dc242e4 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:30.097392 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:30.129449 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f111b3eb-fbb5-440a-8d6f-ee800fcb74a7 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:30.131071 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:30.167253 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da3e36b-e7cb-4881-a62c-b280812572e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:30.167879 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da3e36b-e7cb-4881-a62c-b280812572e1 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:30.168420 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da3e36b-e7cb-4881-a62c-b280812572e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:30.168420 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da3e36b-e7cb-4881-a62c-b280812572e1 None admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:30.169166 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da3e36b-e7cb-4881-a62c-b280812572e1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:02:30.169484 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da3e36b-e7cb-4881-a62c-b280812572e1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:30.170859 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da3e36b-e7cb-4881-a62c-b280812572e1 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:30.175053 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb191a89-d3bc-4896-9fb9-d2f18dc242e4 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:30.189864 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 427/848] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jul 25 12:02:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:02:30.189864 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da3e36b-e7cb-4881-a62c-b280812572e1 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:30.210033 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d3bce9-e89c-4277-a68d-d93e41ab2a42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:30.210729 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d3bce9-e89c-4277-a68d-d93e41ab2a42 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:30.211019 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d3bce9-e89c-4277-a68d-d93e41ab2a42 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:30.211293 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d3bce9-e89c-4277-a68d-d93e41ab2a42 None admin] PATH_INFO: `/v3/services` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:30.212070 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d3bce9-e89c-4277-a68d-d93e41ab2a42 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:02:30.212383 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d3bce9-e89c-4277-a68d-d93e41ab2a42 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:30.213807 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d3bce9-e89c-4277-a68d-d93e41ab2a42 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:30.227587 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 422/849] 10.222.0.42 () {56 vars in 1114 bytes} [Mon Jul 25 12:02:30 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:02:30.227587 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d3bce9-e89c-4277-a68d-d93e41ab2a42 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:30.332946 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da3e36b-e7cb-4881-a62c-b280812572e1 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:30.342138 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:30.357912 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d3bce9-e89c-4277-a68d-d93e41ab2a42 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:30.371496 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5577169-ab86-4491-baf6-82914c7f5479 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:30.372485 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5577169-ab86-4491-baf6-82914c7f5479 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:30.372989 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5577169-ab86-4491-baf6-82914c7f5479 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:30.373338 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5577169-ab86-4491-baf6-82914c7f5479 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:30.383674 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5577169-ab86-4491-baf6-82914c7f5479 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:30.385274 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5577169-ab86-4491-baf6-82914c7f5479 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:30.402210 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-a5577169-ab86-4491-baf6-82914c7f5479 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 25 12:02:30.402210 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:02:30.402210 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:02:30.402210 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:02:30.402210 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:02:30.402210 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:02:30.402210 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:02:30.402210 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:02:30.402210 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:02:30.402210 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:02:30.402210 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:02:30.402210 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:02:30.402210 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 12:02:30.402210 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 12:02:30.402210 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 12:02:30.402210 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 12:02:30.403356 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:02:30.403356 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:02:30.403356 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 12:02:30.403356 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 12:02:30.403356 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 12:02:30.403356 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 12:02:30.403356 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:02:30.403356 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:02:30.403356 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:02:30.403356 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:02:30.403356 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:02:30.403356 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 25 12:02:30.403356 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:02:30.403356 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:02:30.403356 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:02:30.403356 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:02:30.403356 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:02:30.403356 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:02:30.404718 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:02:30.404718 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:02:30.404718 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 12:02:30.404718 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 12:02:30.404718 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 12:02:30.404718 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 12:02:30.404718 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 12:02:30.404718 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 12:02:30.404718 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 25 12:02:30.404718 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 25 12:02:30.405866 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c00f167-7eda-41ba-9a71-79a7c6841e2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:30.406015 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 428/850] 10.222.0.42 () {56 vars in 1117 bytes} [Mon Jul 25 12:02:30 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:02:30.409151 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c00f167-7eda-41ba-9a71-79a7c6841e2d None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:30.409151 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c00f167-7eda-41ba-9a71-79a7c6841e2d None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:30.409151 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c00f167-7eda-41ba-9a71-79a7c6841e2d None admin] PATH_INFO: `/v3/users/swift` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:30.411057 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5577169-ab86-4491-baf6-82914c7f5479 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:30.421000 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c00f167-7eda-41ba-9a71-79a7c6841e2d None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:30.422503 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c00f167-7eda-41ba-9a71-79a7c6841e2d None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:30.439688 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:30.441410 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-3c00f167-7eda-41ba-9a71-79a7c6841e2d None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 25 12:02:30.441410 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:02:30.441410 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:02:30.441410 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:02:30.441410 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:02:30.441410 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:02:30.441410 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:02:30.441410 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:02:30.441410 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:02:30.441410 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:02:30.441410 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:02:30.441410 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:02:30.441410 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 12:02:30.441410 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 12:02:30.441410 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 12:02:30.441410 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 12:02:30.442456 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:02:30.442456 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:02:30.442456 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 12:02:30.442456 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 12:02:30.442456 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 12:02:30.442456 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 12:02:30.442456 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:02:30.442456 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:02:30.442456 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:02:30.442456 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:02:30.442456 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:02:30.442456 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jul 25 12:02:30.442456 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:02:30.442456 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:02:30.442456 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:02:30.442456 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:02:30.442456 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:02:30.442456 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:02:30.443389 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:02:30.443389 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:02:30.443389 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 12:02:30.443389 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 12:02:30.443389 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 12:02:30.443389 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 12:02:30.443389 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 12:02:30.443389 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 12:02:30.443389 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 25 12:02:30.443389 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jul 25 12:02:30.443389 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 423/851] 10.222.0.42 () {56 vars in 1096 bytes} [Mon Jul 25 12:02:30 2022] GET /identity/v3/users/swift => generated 83 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:02:30.448401 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c00f167-7eda-41ba-9a71-79a7c6841e2d None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:30.537484 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5577169-ab86-4491-baf6-82914c7f5479 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:30.569628 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c00f167-7eda-41ba-9a71-79a7c6841e2d None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:30.584005 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10a2fce-0b21-42c6-81d0-60cf200daf30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:30.585111 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10a2fce-0b21-42c6-81d0-60cf200daf30 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:30.585681 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10a2fce-0b21-42c6-81d0-60cf200daf30 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:30.585969 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10a2fce-0b21-42c6-81d0-60cf200daf30 None admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:30.586829 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10a2fce-0b21-42c6-81d0-60cf200daf30 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:02:30.587408 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10a2fce-0b21-42c6-81d0-60cf200daf30 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:30.589638 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10a2fce-0b21-42c6-81d0-60cf200daf30 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:30.612912 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce985605-0447-433f-a1c4-d0db218c0500 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:30.613696 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce985605-0447-433f-a1c4-d0db218c0500 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:30.614055 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce985605-0447-433f-a1c4-d0db218c0500 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:30.614406 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce985605-0447-433f-a1c4-d0db218c0500 None admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:30.615324 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce985605-0447-433f-a1c4-d0db218c0500 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:02:30.615746 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce985605-0447-433f-a1c4-d0db218c0500 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:30.617306 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce985605-0447-433f-a1c4-d0db218c0500 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:30.617486 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 429/852] 10.222.0.42 () {56 vars in 1149 bytes} [Mon Jul 25 12:02:30 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 12:02:30.618742 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10a2fce-0b21-42c6-81d0-60cf200daf30 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:30.643643 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 424/853] 10.222.0.42 () {56 vars in 1135 bytes} [Mon Jul 25 12:02:30 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:02:30.647776 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce985605-0447-433f-a1c4-d0db218c0500 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:30.659277 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:30.715690 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:30.737688 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10a2fce-0b21-42c6-81d0-60cf200daf30 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:30.756622 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce985605-0447-433f-a1c4-d0db218c0500 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:30.774838 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8062f3-2b9a-4de7-be5d-bcc45a359108 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:30.775586 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8062f3-2b9a-4de7-be5d-bcc45a359108 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:30.777937 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8062f3-2b9a-4de7-be5d-bcc45a359108 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:30.778389 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8062f3-2b9a-4de7-be5d-bcc45a359108 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:30.798989 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6407df72-e7d3-44ad-bbef-e1d3bac99741 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:30.800252 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6407df72-e7d3-44ad-bbef-e1d3bac99741 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:30.800894 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6407df72-e7d3-44ad-bbef-e1d3bac99741 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:30.801516 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6407df72-e7d3-44ad-bbef-e1d3bac99741 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:30.853360 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8062f3-2b9a-4de7-be5d-bcc45a359108 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:30.855075 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8062f3-2b9a-4de7-be5d-bcc45a359108 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:30.874643 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6407df72-e7d3-44ad-bbef-e1d3bac99741 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:30.876308 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6407df72-e7d3-44ad-bbef-e1d3bac99741 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:30.887277 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:30.941805 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:30.967952 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 430/854] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Jul 25 12:02:30 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 25 12:02:30.969746 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8062f3-2b9a-4de7-be5d-bcc45a359108 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:31.022691 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 425/855] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Jul 25 12:02:30 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 25 12:02:31.025177 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6407df72-e7d3-44ad-bbef-e1d3bac99741 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:31.092240 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8062f3-2b9a-4de7-be5d-bcc45a359108 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:31.129912 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f1272b-710e-427f-b658-d67d962f7cf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:31.130706 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f1272b-710e-427f-b658-d67d962f7cf3 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:31.130988 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f1272b-710e-427f-b658-d67d962f7cf3 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:31.131264 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f1272b-710e-427f-b658-d67d962f7cf3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:31.141317 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f1272b-710e-427f-b658-d67d962f7cf3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:31.142787 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f1272b-710e-427f-b658-d67d962f7cf3 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:31.154497 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6407df72-e7d3-44ad-bbef-e1d3bac99741 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:31.160332 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-78f1272b-710e-427f-b658-d67d962f7cf3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:02:31.160332 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:02:31.160332 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:02:31.160332 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:02:31.160332 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:02:31.160332 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:02:31.160332 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:02:31.160332 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:02:31.160332 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:02:31.160332 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:02:31.160332 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:02:31.160332 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:02:31.160332 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 12:02:31.160332 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 12:02:31.160332 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 12:02:31.161226 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 12:02:31.161226 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:02:31.161226 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:02:31.161226 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:02:31.161226 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:02:31.161226 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:02:31.161226 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:02:31.161226 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:02:31.161226 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 25 12:02:31.161226 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:02:31.161226 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:02:31.161226 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:02:31.161226 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:02:31.161226 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:02:31.161226 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:02:31.161226 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:02:31.161226 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:02:31.162125 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 12:02:31.162125 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 12:02:31.162125 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:02:31.162125 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 25 12:02:31.162125 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 431/856] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jul 25 12:02:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:02:31.163084 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b20efa-2e15-481b-9fe7-f622ca02120b None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:31.163372 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b20efa-2e15-481b-9fe7-f622ca02120b None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:31.163675 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b20efa-2e15-481b-9fe7-f622ca02120b None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:31.169150 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 432/857] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 25 12:02:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:02:31.169150 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b20efa-2e15-481b-9fe7-f622ca02120b None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:31.171586 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:31.200689 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6232ccff-91de-4ab9-917f-c11d33ccc3fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:31.201487 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6232ccff-91de-4ab9-917f-c11d33ccc3fe None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:31.201639 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:31.201773 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6232ccff-91de-4ab9-917f-c11d33ccc3fe None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:31.202060 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6232ccff-91de-4ab9-917f-c11d33ccc3fe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:31.212690 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6232ccff-91de-4ab9-917f-c11d33ccc3fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:31.214126 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6232ccff-91de-4ab9-917f-c11d33ccc3fe None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:31.231890 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-6232ccff-91de-4ab9-917f-c11d33ccc3fe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:02:31.231890 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:02:31.231890 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:02:31.231890 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:02:31.231890 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:02:31.231890 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:02:31.231890 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:02:31.231890 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:02:31.231890 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:02:31.231890 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:02:31.231890 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:02:31.231890 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:02:31.231890 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 12:02:31.231890 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 12:02:31.231890 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 12:02:31.232791 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 12:02:31.232791 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:02:31.232791 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:02:31.232791 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:02:31.232791 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:02:31.232791 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:02:31.232791 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:02:31.232791 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:02:31.232791 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jul 25 12:02:31.232791 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:02:31.232791 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:02:31.232791 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:02:31.232791 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:02:31.232791 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:02:31.232791 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:02:31.232791 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:02:31.232791 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:02:31.233687 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 12:02:31.233687 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 12:02:31.233687 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:02:31.233687 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jul 25 12:02:31.233687 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 426/858] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jul 25 12:02:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:02:31.234296 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042bf59d-840e-49d8-99eb-d002956d7666 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:31.234463 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042bf59d-840e-49d8-99eb-d002956d7666 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:31.234743 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042bf59d-840e-49d8-99eb-d002956d7666 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:31.252367 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-042bf59d-840e-49d8-99eb-d002956d7666 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:02:31.253560 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-042bf59d-840e-49d8-99eb-d002956d7666 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:02:31.284393 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b20efa-2e15-481b-9fe7-f622ca02120b None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:31.321754 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 427/859] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 25 12:02:31 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 12:02:31.322143 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d709f309-36d4-40c0-aaa4-82d75b4e2366 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:31.322895 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d709f309-36d4-40c0-aaa4-82d75b4e2366 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:31.323257 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d709f309-36d4-40c0-aaa4-82d75b4e2366 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:31.323518 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042bf59d-840e-49d8-99eb-d002956d7666 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:31.323713 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d709f309-36d4-40c0-aaa4-82d75b4e2366 None admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:31.324457 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d709f309-36d4-40c0-aaa4-82d75b4e2366 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:02:31.324840 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d709f309-36d4-40c0-aaa4-82d75b4e2366 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:31.326299 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d709f309-36d4-40c0-aaa4-82d75b4e2366 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:31.342260 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 433/860] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jul 25 12:02:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 12:02:31.343173 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d709f309-36d4-40c0-aaa4-82d75b4e2366 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:31.428235 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:31.440676 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:31.445450 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042bf59d-840e-49d8-99eb-d002956d7666 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:31.451796 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d709f309-36d4-40c0-aaa4-82d75b4e2366 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:31.483617 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27884756-77d3-4976-9872-98dc9366482d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:31.484214 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27884756-77d3-4976-9872-98dc9366482d None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:31.484525 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27884756-77d3-4976-9872-98dc9366482d None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:31.484778 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27884756-77d3-4976-9872-98dc9366482d None admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:31.485530 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27884756-77d3-4976-9872-98dc9366482d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:02:31.485840 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27884756-77d3-4976-9872-98dc9366482d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:31.486961 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f220bc-df1a-4a8d-8e29-95cab20367c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:31.487216 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27884756-77d3-4976-9872-98dc9366482d None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:31.487804 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f220bc-df1a-4a8d-8e29-95cab20367c7 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:31.488097 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f220bc-df1a-4a8d-8e29-95cab20367c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:31.488382 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f220bc-df1a-4a8d-8e29-95cab20367c7 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:31.489035 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f220bc-df1a-4a8d-8e29-95cab20367c7 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:31.490393 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f220bc-df1a-4a8d-8e29-95cab20367c7 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:31.504139 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 428/861] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jul 25 12:02:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:02:31.504139 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27884756-77d3-4976-9872-98dc9366482d None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:31.504377 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-91f220bc-df1a-4a8d-8e29-95cab20367c7 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 25 12:02:31.504377 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:02:31.504377 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 12:02:31.504377 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 12:02:31.504377 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 12:02:31.504377 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 12:02:31.504377 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 12:02:31.504377 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 12:02:31.504377 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 25 12:02:31.504377 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 25 12:02:31.504377 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 12:02:31.504377 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 25 12:02:31.504377 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:02:31.504377 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:02:31.504377 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:02:31.504377 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:02:31.505972 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:02:31.505972 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:02:31.505972 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:02:31.505972 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:02:31.505972 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:02:31.505972 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:02:31.505972 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:02:31.505972 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 12:02:31.505972 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 12:02:31.505972 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 12:02:31.505972 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 12:02:31.505972 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:02:31.505972 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:02:31.505972 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:02:31.505972 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:02:31.505972 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:02:31.506810 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:02:31.506810 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:02:31.506810 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 25 12:02:31.506810 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:02:31.506810 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:02:31.506810 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:02:31.506810 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:02:31.506810 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:02:31.506810 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:02:31.506810 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:02:31.506810 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:02:31.506810 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 12:02:31.506810 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 12:02:31.506810 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 25 12:02:31.506810 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 25 12:02:31.506810 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 434/862] 10.222.0.42 () {56 vars in 1132 bytes} [Mon Jul 25 12:02:31 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:02:31.510107 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f220bc-df1a-4a8d-8e29-95cab20367c7 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:31.611285 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27884756-77d3-4976-9872-98dc9366482d None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:31.616206 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f220bc-df1a-4a8d-8e29-95cab20367c7 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:31.647502 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a41bdd-0d1e-4b49-8dff-40b9cce95889 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:31.648329 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a41bdd-0d1e-4b49-8dff-40b9cce95889 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:31.648612 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a41bdd-0d1e-4b49-8dff-40b9cce95889 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:31.648890 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a41bdd-0d1e-4b49-8dff-40b9cce95889 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:31.652357 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8e7791-8121-444c-b778-2940f6ee25c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:31.653357 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8e7791-8121-444c-b778-2940f6ee25c7 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:31.653870 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8e7791-8121-444c-b778-2940f6ee25c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:31.654351 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8e7791-8121-444c-b778-2940f6ee25c7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:31.655775 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a41bdd-0d1e-4b49-8dff-40b9cce95889 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:31.657785 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a41bdd-0d1e-4b49-8dff-40b9cce95889 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:31.662516 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8e7791-8121-444c-b778-2940f6ee25c7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:31.664177 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8e7791-8121-444c-b778-2940f6ee25c7 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:31.672027 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-87a41bdd-0d1e-4b49-8dff-40b9cce95889 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 12:02:31.672027 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:02:31.672027 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:02:31.672027 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:02:31.672027 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:02:31.672027 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:02:31.672027 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:02:31.672027 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:02:31.672027 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:02:31.672027 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:02:31.672027 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:02:31.672027 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:02:31.672027 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 12:02:31.672027 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 12:02:31.672027 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 12:02:31.672931 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 12:02:31.672931 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:02:31.672931 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:02:31.672931 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:02:31.672931 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:02:31.672931 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:02:31.672931 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:02:31.672931 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:02:31.672931 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jul 25 12:02:31.672931 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:02:31.672931 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:02:31.672931 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:02:31.672931 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:02:31.672931 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:02:31.672931 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:02:31.672931 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:02:31.672931 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:02:31.673972 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 12:02:31.673972 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 12:02:31.673972 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 12:02:31.673972 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 12:02:31.673972 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 12:02:31.673972 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 12:02:31.673972 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 12:02:31.673972 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jul 25 12:02:31.673972 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 429/863] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Jul 25 12:02:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:02:31.674518 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a41bdd-0d1e-4b49-8dff-40b9cce95889 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:31.681778 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-0b8e7791-8121-444c-b778-2940f6ee25c7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 12:02:31.681778 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:02:31.681778 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:02:31.681778 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:02:31.681778 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:02:31.681778 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:02:31.681778 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:02:31.681778 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:02:31.681778 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:02:31.681778 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:02:31.681778 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:02:31.681778 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:02:31.681778 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 12:02:31.681778 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 12:02:31.681778 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 12:02:31.682818 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 12:02:31.682818 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:02:31.682818 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:02:31.682818 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:02:31.682818 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:02:31.682818 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:02:31.682818 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:02:31.682818 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:02:31.682818 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 25 12:02:31.682818 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:02:31.682818 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:02:31.682818 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:02:31.682818 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:02:31.682818 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:02:31.682818 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:02:31.682818 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:02:31.682818 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:02:31.683925 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 12:02:31.683925 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 12:02:31.683925 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 12:02:31.683925 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 12:02:31.683925 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 12:02:31.683925 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 12:02:31.683925 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 12:02:31.683925 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 25 12:02:31.690250 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:31.690496 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 435/864] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Jul 25 12:02:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:02:31.691535 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8e7791-8121-444c-b778-2940f6ee25c7 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:31.711889 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:31.780586 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a41bdd-0d1e-4b49-8dff-40b9cce95889 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:31.817061 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc492d9-0140-40f3-8366-4bc4d5f557d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:31.817513 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8e7791-8121-444c-b778-2940f6ee25c7 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:31.817726 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc492d9-0140-40f3-8366-4bc4d5f557d1 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:31.818014 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc492d9-0140-40f3-8366-4bc4d5f557d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:31.818294 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc492d9-0140-40f3-8366-4bc4d5f557d1 None admin] PATH_INFO: `/v3/services` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:31.818997 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc492d9-0140-40f3-8366-4bc4d5f557d1 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:02:31.819312 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc492d9-0140-40f3-8366-4bc4d5f557d1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:31.820723 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc492d9-0140-40f3-8366-4bc4d5f557d1 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:31.830636 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 430/865] 10.222.0.42 () {56 vars in 1126 bytes} [Mon Jul 25 12:02:31 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:02:31.830636 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc492d9-0140-40f3-8366-4bc4d5f557d1 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:31.855822 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d644f10b-edb3-4b88-9756-f426b3ca9c1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:31.856491 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d644f10b-edb3-4b88-9756-f426b3ca9c1d None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:31.856784 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d644f10b-edb3-4b88-9756-f426b3ca9c1d None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:31.857066 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d644f10b-edb3-4b88-9756-f426b3ca9c1d None admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:31.857895 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d644f10b-edb3-4b88-9756-f426b3ca9c1d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:02:31.858163 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d644f10b-edb3-4b88-9756-f426b3ca9c1d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:31.859613 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d644f10b-edb3-4b88-9756-f426b3ca9c1d None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:31.872350 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 436/866] 10.222.0.42 () {56 vars in 1148 bytes} [Mon Jul 25 12:02:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:02:31.873449 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d644f10b-edb3-4b88-9756-f426b3ca9c1d None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:31.920640 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:31.948201 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc492d9-0140-40f3-8366-4bc4d5f557d1 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:31.957493 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:31.984140 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdfc3ee-fa43-4e8d-ad0f-bd8333545fdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:31.984837 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdfc3ee-fa43-4e8d-ad0f-bd8333545fdd None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:31.985129 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdfc3ee-fa43-4e8d-ad0f-bd8333545fdd None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:31.985620 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdfc3ee-fa43-4e8d-ad0f-bd8333545fdd None admin] PATH_INFO: `/v3/projects` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:31.985726 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d644f10b-edb3-4b88-9756-f426b3ca9c1d None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:31.986192 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bdfc3ee-fa43-4e8d-ad0f-bd8333545fdd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:02:31.986508 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bdfc3ee-fa43-4e8d-ad0f-bd8333545fdd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:31.987970 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bdfc3ee-fa43-4e8d-ad0f-bd8333545fdd None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:32.000003 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 431/867] 10.222.0.42 () {56 vars in 1148 bytes} [Mon Jul 25 12:02:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:02:32.001313 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdfc3ee-fa43-4e8d-ad0f-bd8333545fdd None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:32.029447 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e210b5-ebf1-47df-b85b-5dc19cef5c59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:32.030044 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e210b5-ebf1-47df-b85b-5dc19cef5c59 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:32.030334 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e210b5-ebf1-47df-b85b-5dc19cef5c59 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:32.030613 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e210b5-ebf1-47df-b85b-5dc19cef5c59 None admin] PATH_INFO: `/v3/services` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:32.031330 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e210b5-ebf1-47df-b85b-5dc19cef5c59 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:02:32.031676 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e210b5-ebf1-47df-b85b-5dc19cef5c59 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:32.033081 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e210b5-ebf1-47df-b85b-5dc19cef5c59 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:32.047930 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 437/868] 10.222.0.42 () {56 vars in 1126 bytes} [Mon Jul 25 12:02:31 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:02:32.047930 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e210b5-ebf1-47df-b85b-5dc19cef5c59 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:32.122864 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdfc3ee-fa43-4e8d-ad0f-bd8333545fdd None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:32.168976 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0884b483-cd64-441d-ac0b-7ab08ba545cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:32.169971 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0884b483-cd64-441d-ac0b-7ab08ba545cc None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:32.170267 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0884b483-cd64-441d-ac0b-7ab08ba545cc None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:32.170553 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0884b483-cd64-441d-ac0b-7ab08ba545cc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:32.171405 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0884b483-cd64-441d-ac0b-7ab08ba545cc None admin] RBAC: Adding query filter params (role.id=79dec36aee5b4322bf652041c25ca6e4, scope.project.id=b9f62b8f37304b8a9a4bb54205347577, user.id=93207b858d294d049fa56d45810a40aa) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:02:32.171757 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0884b483-cd64-441d-ac0b-7ab08ba545cc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:32.174321 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0884b483-cd64-441d-ac0b-7ab08ba545cc None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:32.177018 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:32.187803 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 432/869] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Jul 25 12:02:32 2022] GET /identity/v3/role_assignments?user.id=93207b858d294d049fa56d45810a40aa&scope.project.id=b9f62b8f37304b8a9a4bb54205347577&role.id=79dec36aee5b4322bf652041c25ca6e4 => generated 608 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:02:32.194326 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e210b5-ebf1-47df-b85b-5dc19cef5c59 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:32.242692 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b4043e-f85b-436d-8cd4-d54f904ac8f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:32.243394 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b4043e-f85b-436d-8cd4-d54f904ac8f7 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:32.243756 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b4043e-f85b-436d-8cd4-d54f904ac8f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:32.244042 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b4043e-f85b-436d-8cd4-d54f904ac8f7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:32.244905 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b4043e-f85b-436d-8cd4-d54f904ac8f7 None admin] RBAC: Adding query filter params (role.id=a7d20885b6d94928a34b3cc5b3a00006, scope.project.id=b9f62b8f37304b8a9a4bb54205347577, user.id=7bd782ea66364eeb96592096eae63d4f) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:02:32.245223 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b4043e-f85b-436d-8cd4-d54f904ac8f7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:32.246707 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b4043e-f85b-436d-8cd4-d54f904ac8f7 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:32.259753 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 438/870] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Jul 25 12:02:32 2022] GET /identity/v3/role_assignments?user.id=7bd782ea66364eeb96592096eae63d4f&scope.project.id=b9f62b8f37304b8a9a4bb54205347577&role.id=a7d20885b6d94928a34b3cc5b3a00006 => generated 608 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:02:32.364181 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408e32f0-8fe2-4120-b68a-8dab5eb15ccb None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:32.364540 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408e32f0-8fe2-4120-b68a-8dab5eb15ccb None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:32.364755 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408e32f0-8fe2-4120-b68a-8dab5eb15ccb None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:32.366485 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 433/871] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 25 12:02:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:02:32.371824 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919215d6-5531-4ad0-9b9b-0e0d5767efe1 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:32.372323 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919215d6-5531-4ad0-9b9b-0e0d5767efe1 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:32.372428 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919215d6-5531-4ad0-9b9b-0e0d5767efe1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:32.381458 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:32.395645 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-919215d6-5531-4ad0-9b9b-0e0d5767efe1 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:02:32.396906 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-919215d6-5531-4ad0-9b9b-0e0d5767efe1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:02:32.474186 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 439/872] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 25 12:02:32 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 12:02:32.509475 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408e32f0-8fe2-4120-b68a-8dab5eb15ccb None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:32.684870 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408e32f0-8fe2-4120-b68a-8dab5eb15ccb None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:32.721257 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f35fd2-d928-4d2c-a740-9d309f8e8345 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:32.721935 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f35fd2-d928-4d2c-a740-9d309f8e8345 None admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:32.722228 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f35fd2-d928-4d2c-a740-9d309f8e8345 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:32.722511 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f35fd2-d928-4d2c-a740-9d309f8e8345 None admin] PATH_INFO: `/v3/services` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:32.723246 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f35fd2-d928-4d2c-a740-9d309f8e8345 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:32.724663 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f35fd2-d928-4d2c-a740-9d309f8e8345 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:32.743904 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 434/873] 10.222.0.42 () {60 vars in 1141 bytes} [Mon Jul 25 12:02:32 2022] POST /identity/v3/services => generated 251 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:02:34.106103 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fc2210-2830-4170-8882-b485f059b0f9 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:34.107128 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fc2210-2830-4170-8882-b485f059b0f9 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:34.107649 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fc2210-2830-4170-8882-b485f059b0f9 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:34.109446 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 440/874] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 25 12:02:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:02:34.115019 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49f510f-4f5e-44a7-8696-f875a589dffc None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:34.115873 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49f510f-4f5e-44a7-8696-f875a589dffc None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:34.116358 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49f510f-4f5e-44a7-8696-f875a589dffc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:34.135509 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-a49f510f-4f5e-44a7-8696-f875a589dffc None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:02:34.137032 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-a49f510f-4f5e-44a7-8696-f875a589dffc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:02:34.193837 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:34.216633 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 435/875] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 25 12:02:34 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 25 12:02:34.238825 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fc2210-2830-4170-8882-b485f059b0f9 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:34.333389 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f446bda-e796-4c9a-8cd9-83096cb15324 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:34.334029 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f446bda-e796-4c9a-8cd9-83096cb15324 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:34.334529 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f446bda-e796-4c9a-8cd9-83096cb15324 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:34.336382 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 436/876] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 25 12:02:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 12:02:34.341104 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d5fcc1-735d-4562-a508-b3db02ae8f67 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:34.341635 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d5fcc1-735d-4562-a508-b3db02ae8f67 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:34.342133 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d5fcc1-735d-4562-a508-b3db02ae8f67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:34.349483 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fc2210-2830-4170-8882-b485f059b0f9 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:34.360561 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-05d5fcc1-735d-4562-a508-b3db02ae8f67 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:02:34.362041 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-05d5fcc1-735d-4562-a508-b3db02ae8f67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:02:34.384668 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91081f3c-9fd9-48d0-99a5-aa12f78cf2ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:34.385332 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91081f3c-9fd9-48d0-99a5-aa12f78cf2ef None admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:34.385786 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91081f3c-9fd9-48d0-99a5-aa12f78cf2ef None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:34.386240 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91081f3c-9fd9-48d0-99a5-aa12f78cf2ef None admin] PATH_INFO: `/v3/services` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:34.387123 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91081f3c-9fd9-48d0-99a5-aa12f78cf2ef None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:34.388688 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91081f3c-9fd9-48d0-99a5-aa12f78cf2ef None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:34.392897 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:34.411582 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 441/877] 10.222.0.42 () {60 vars in 1141 bytes} [Mon Jul 25 12:02:34 2022] POST /identity/v3/services => generated 269 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:02:34.455691 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 437/878] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 25 12:02:34 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 25 12:02:34.474838 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91081f3c-9fd9-48d0-99a5-aa12f78cf2ef None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:34.492363 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2eb1c97-d01d-4cd3-9487-8aedb9092dce None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:34.492673 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2eb1c97-d01d-4cd3-9487-8aedb9092dce None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:34.492929 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2eb1c97-d01d-4cd3-9487-8aedb9092dce None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:34.494528 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 438/879] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 25 12:02:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 12:02:34.500120 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54621cb1-3a66-4185-91f7-cb95960f9732 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:34.500681 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54621cb1-3a66-4185-91f7-cb95960f9732 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:34.501651 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54621cb1-3a66-4185-91f7-cb95960f9732 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:34.522822 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-54621cb1-3a66-4185-91f7-cb95960f9732 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:02:34.524324 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-54621cb1-3a66-4185-91f7-cb95960f9732 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:02:34.610762 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91081f3c-9fd9-48d0-99a5-aa12f78cf2ef None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:34.615631 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 439/880] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 25 12:02:34 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 25 12:02:34.637834 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54621cb1-3a66-4185-91f7-cb95960f9732 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:34.652123 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4495d117-d08f-4658-bb2b-2b6235e5c2fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:34.653272 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4495d117-d08f-4658-bb2b-2b6235e5c2fb None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:34.653791 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4495d117-d08f-4658-bb2b-2b6235e5c2fb None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:34.654276 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4495d117-d08f-4658-bb2b-2b6235e5c2fb None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:34.665340 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4495d117-d08f-4658-bb2b-2b6235e5c2fb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 12:02:34.674561 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-4495d117-d08f-4658-bb2b-2b6235e5c2fb None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 25 12:02:34.674561 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:02:34.674561 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:02:34.674561 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:02:34.674561 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:02:34.674561 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:02:34.674561 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:02:34.674561 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:02:34.674561 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:02:34.674561 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:02:34.674561 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:02:34.674561 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:02:34.674561 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 12:02:34.674561 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 12:02:34.674561 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 12:02:34.674561 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 12:02:34.675512 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 12:02:34.675512 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 12:02:34.675512 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 12:02:34.675512 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 12:02:34.675512 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:02:34.675512 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:02:34.675512 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:02:34.675512 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:02:34.675512 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:02:34.675512 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:02:34.675512 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:02:34.675512 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 25 12:02:34.675512 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:02:34.675512 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:02:34.675512 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:02:34.675512 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:02:34.676390 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:02:34.676390 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:02:34.676390 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:02:34.676390 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:02:34.676390 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 12:02:34.676390 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 12:02:34.676390 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 12:02:34.676390 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 12:02:34.676390 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 12:02:34.676390 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 12:02:34.676390 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 25 12:02:34.676390 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 25 12:02:34.683740 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 442/881] 10.222.0.42 () {56 vars in 1120 bytes} [Mon Jul 25 12:02:34 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:02:34.685222 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4495d117-d08f-4658-bb2b-2b6235e5c2fb None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:34.690197 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:34.742319 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54621cb1-3a66-4185-91f7-cb95960f9732 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:34.743006 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:34.781400 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5e3728-36fc-4287-87ef-afb7b7dbeddb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:34.782472 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5e3728-36fc-4287-87ef-afb7b7dbeddb None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:34.782934 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5e3728-36fc-4287-87ef-afb7b7dbeddb None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:34.783384 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5e3728-36fc-4287-87ef-afb7b7dbeddb None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:34.784252 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5e3728-36fc-4287-87ef-afb7b7dbeddb None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:34.785876 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5e3728-36fc-4287-87ef-afb7b7dbeddb None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:34.799577 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-fa5e3728-36fc-4287-87ef-afb7b7dbeddb None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 25 12:02:34.799577 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:02:34.799577 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 12:02:34.799577 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 12:02:34.799577 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 12:02:34.799577 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 12:02:34.799577 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 12:02:34.799577 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 12:02:34.799577 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 25 12:02:34.799577 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jul 25 12:02:34.799577 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 12:02:34.799577 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jul 25 12:02:34.799577 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:02:34.799577 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:02:34.799577 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:02:34.799577 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:02:34.799577 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:02:34.800576 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:02:34.800576 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:02:34.800576 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:02:34.800576 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:02:34.800576 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:02:34.800576 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:02:34.800576 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 12:02:34.800576 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 12:02:34.800576 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 12:02:34.800576 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 12:02:34.800576 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:02:34.800576 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:02:34.800576 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:02:34.800576 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:02:34.800576 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:02:34.800576 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:02:34.801397 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:02:34.801397 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jul 25 12:02:34.801397 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:02:34.801397 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:02:34.801397 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:02:34.801397 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:02:34.801397 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:02:34.801397 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:02:34.801397 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:02:34.801397 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:02:34.801397 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 12:02:34.801397 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 12:02:34.801397 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 25 12:02:34.801397 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jul 25 12:02:34.802127 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4495d117-d08f-4658-bb2b-2b6235e5c2fb None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:34.804459 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 440/882] 10.222.0.42 () {56 vars in 1126 bytes} [Mon Jul 25 12:02:34 2022] GET /identity/v3/services/object-store => generated 93 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 12:02:34.812541 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5e3728-36fc-4287-87ef-afb7b7dbeddb None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:34.846525 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f223c31-79a9-4fe9-8261-948d14937c73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:34.847191 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f223c31-79a9-4fe9-8261-948d14937c73 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:34.847700 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f223c31-79a9-4fe9-8261-948d14937c73 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:34.847984 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f223c31-79a9-4fe9-8261-948d14937c73 None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:34.848728 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f223c31-79a9-4fe9-8261-948d14937c73 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:02:34.849037 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f223c31-79a9-4fe9-8261-948d14937c73 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:34.850421 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f223c31-79a9-4fe9-8261-948d14937c73 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:34.858645 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 443/883] 10.222.0.42 () {56 vars in 1115 bytes} [Mon Jul 25 12:02:34 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 12:02:34.860637 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dd0f12-6c85-4962-81ce-a17bbf4a5e74 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:34.861129 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dd0f12-6c85-4962-81ce-a17bbf4a5e74 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:34.861557 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dd0f12-6c85-4962-81ce-a17bbf4a5e74 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:34.863151 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 444/884] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 25 12:02:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:02:34.865595 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64dd0f12-6c85-4962-81ce-a17bbf4a5e74 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:34.922465 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5e3728-36fc-4287-87ef-afb7b7dbeddb None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:34.926766 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:34.962311 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31a41e7-c5a7-4d79-88e4-88287a96f332 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:34.963074 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31a41e7-c5a7-4d79-88e4-88287a96f332 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:34.963367 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31a41e7-c5a7-4d79-88e4-88287a96f332 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:34.963688 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31a41e7-c5a7-4d79-88e4-88287a96f332 None admin] PATH_INFO: `/v3/services` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:34.964411 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31a41e7-c5a7-4d79-88e4-88287a96f332 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:02:34.964730 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31a41e7-c5a7-4d79-88e4-88287a96f332 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:34.966109 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31a41e7-c5a7-4d79-88e4-88287a96f332 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:34.978225 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 441/885] 10.222.0.42 () {56 vars in 1122 bytes} [Mon Jul 25 12:02:34 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:02:34.978225 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f64a1e-d270-46b4-b791-821760f238e5 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:34.978225 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f64a1e-d270-46b4-b791-821760f238e5 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:34.978225 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f64a1e-d270-46b4-b791-821760f238e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:34.993327 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-39f64a1e-d270-46b4-b791-821760f238e5 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:02:34.994530 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-39f64a1e-d270-46b4-b791-821760f238e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:02:34.999813 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64dd0f12-6c85-4962-81ce-a17bbf4a5e74 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:35.018156 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:35.037952 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2454d730-e084-447f-bb01-02ef9ace235e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:35.039001 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2454d730-e084-447f-bb01-02ef9ace235e None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:35.039465 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2454d730-e084-447f-bb01-02ef9ace235e None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:35.039982 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2454d730-e084-447f-bb01-02ef9ace235e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:35.053380 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2454d730-e084-447f-bb01-02ef9ace235e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:35.056673 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2454d730-e084-447f-bb01-02ef9ace235e None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:35.068927 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 442/886] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 25 12:02:34 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 12:02:35.069707 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f64a1e-d270-46b4-b791-821760f238e5 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:35.085451 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-2454d730-e084-447f-bb01-02ef9ace235e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:02:35.085451 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:02:35.085451 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:02:35.085451 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:02:35.085451 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:02:35.085451 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:02:35.085451 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:02:35.085451 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:02:35.085451 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:02:35.085451 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:02:35.085451 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:02:35.085451 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:02:35.085451 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 12:02:35.085451 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 12:02:35.085451 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 12:02:35.086382 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 12:02:35.086382 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:02:35.086382 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:02:35.086382 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:02:35.086382 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:02:35.086382 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:02:35.086382 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:02:35.086382 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:02:35.086382 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 25 12:02:35.086382 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:02:35.086382 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:02:35.086382 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:02:35.086382 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:02:35.086382 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:02:35.086382 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:02:35.086382 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:02:35.086382 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:02:35.092477 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 12:02:35.092477 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 12:02:35.092477 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:02:35.092477 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 25 12:02:35.092477 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 445/887] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jul 25 12:02:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:02:35.095363 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2454d730-e084-447f-bb01-02ef9ace235e None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:35.183932 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:35.188467 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f64a1e-d270-46b4-b791-821760f238e5 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:35.200765 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2454d730-e084-447f-bb01-02ef9ace235e None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:35.226085 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4507ef4-8c1a-4502-b502-5f3fed421830 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:35.227113 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4507ef4-8c1a-4502-b502-5f3fed421830 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:35.227709 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4507ef4-8c1a-4502-b502-5f3fed421830 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:35.228158 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4507ef4-8c1a-4502-b502-5f3fed421830 None admin] PATH_INFO: `/v3/services` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:35.229014 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4507ef4-8c1a-4502-b502-5f3fed421830 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:02:35.229568 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4507ef4-8c1a-4502-b502-5f3fed421830 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:35.231185 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4507ef4-8c1a-4502-b502-5f3fed421830 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:35.236504 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6516913-0a7e-4f3f-9e3f-8585973aabbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:35.247635 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6516913-0a7e-4f3f-9e3f-8585973aabbf None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:35.247853 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 443/888] 10.222.0.42 () {56 vars in 1122 bytes} [Mon Jul 25 12:02:35 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:02:35.247950 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6516913-0a7e-4f3f-9e3f-8585973aabbf None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:35.247950 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6516913-0a7e-4f3f-9e3f-8585973aabbf None admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:35.248108 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4507ef4-8c1a-4502-b502-5f3fed421830 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:35.248219 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6516913-0a7e-4f3f-9e3f-8585973aabbf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:02:35.248219 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6516913-0a7e-4f3f-9e3f-8585973aabbf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:35.248219 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6516913-0a7e-4f3f-9e3f-8585973aabbf None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:35.254501 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 446/889] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jul 25 12:02:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:02:35.260634 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6516913-0a7e-4f3f-9e3f-8585973aabbf None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:35.293994 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:35.362959 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4507ef4-8c1a-4502-b502-5f3fed421830 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:35.376603 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6516913-0a7e-4f3f-9e3f-8585973aabbf None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:35.402494 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe140e4c-6a9b-442b-aa72-daa9ca1ba796 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:35.403769 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe140e4c-6a9b-442b-aa72-daa9ca1ba796 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:35.404244 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe140e4c-6a9b-442b-aa72-daa9ca1ba796 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:35.404676 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe140e4c-6a9b-442b-aa72-daa9ca1ba796 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:35.405502 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe140e4c-6a9b-442b-aa72-daa9ca1ba796 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:35.407877 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe140e4c-6a9b-442b-aa72-daa9ca1ba796 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:35.415851 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188f942e-2719-4b16-af6b-17324d8cedff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:35.416683 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188f942e-2719-4b16-af6b-17324d8cedff None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:35.416969 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188f942e-2719-4b16-af6b-17324d8cedff None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:35.417249 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188f942e-2719-4b16-af6b-17324d8cedff None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:35.422336 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-fe140e4c-6a9b-442b-aa72-daa9ca1ba796 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 25 12:02:35.422336 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:02:35.422336 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 12:02:35.422336 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 12:02:35.422336 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 12:02:35.422336 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 12:02:35.422336 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 12:02:35.422336 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 12:02:35.422336 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 25 12:02:35.422336 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jul 25 12:02:35.422336 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 12:02:35.422336 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jul 25 12:02:35.422336 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:02:35.422336 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:02:35.422336 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:02:35.422336 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:02:35.422336 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:02:35.423367 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:02:35.423367 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:02:35.423367 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:02:35.423367 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:02:35.423367 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:02:35.423367 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:02:35.423367 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 12:02:35.423367 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 12:02:35.423367 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 12:02:35.423367 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 12:02:35.423367 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:02:35.423367 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:02:35.423367 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:02:35.423367 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:02:35.423367 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:02:35.423367 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:02:35.424257 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:02:35.424257 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jul 25 12:02:35.424257 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:02:35.424257 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:02:35.424257 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:02:35.424257 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:02:35.424257 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:02:35.424257 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:02:35.424257 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:02:35.424257 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:02:35.424257 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 12:02:35.424257 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 12:02:35.424257 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 25 12:02:35.424257 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jul 25 12:02:35.425242 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188f942e-2719-4b16-af6b-17324d8cedff None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:35.426845 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188f942e-2719-4b16-af6b-17324d8cedff None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:35.427658 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 444/890] 10.222.0.42 () {56 vars in 1114 bytes} [Mon Jul 25 12:02:35 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:02:35.429834 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe140e4c-6a9b-442b-aa72-daa9ca1ba796 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:35.444409 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-188f942e-2719-4b16-af6b-17324d8cedff None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 25 12:02:35.444409 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:02:35.444409 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:02:35.444409 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:02:35.444409 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:02:35.444409 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:02:35.444409 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:02:35.444409 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:02:35.444409 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:02:35.444409 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:02:35.444409 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:02:35.444409 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:02:35.444409 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 12:02:35.444409 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 12:02:35.444409 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 12:02:35.444409 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 12:02:35.445492 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:02:35.445492 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:02:35.445492 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 12:02:35.445492 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 12:02:35.445492 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 12:02:35.445492 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 12:02:35.445492 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:02:35.445492 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:02:35.445492 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:02:35.445492 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:02:35.445492 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:02:35.445492 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 25 12:02:35.445492 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:02:35.445492 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:02:35.445492 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:02:35.445492 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:02:35.445492 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:02:35.445492 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:02:35.446448 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:02:35.446448 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:02:35.446448 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 12:02:35.446448 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 12:02:35.446448 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 12:02:35.446448 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 12:02:35.446448 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 12:02:35.446448 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 12:02:35.446448 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 25 12:02:35.446448 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 25 12:02:35.448347 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 447/891] 10.222.0.42 () {56 vars in 1117 bytes} [Mon Jul 25 12:02:35 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 12:02:35.455384 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188f942e-2719-4b16-af6b-17324d8cedff None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:35.463178 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:35.546198 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe140e4c-6a9b-442b-aa72-daa9ca1ba796 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:35.580481 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:35.583699 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9535d8-72de-4440-9ae4-d3e000daab17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:35.584601 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9535d8-72de-4440-9ae4-d3e000daab17 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:35.585094 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9535d8-72de-4440-9ae4-d3e000daab17 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:35.585577 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9535d8-72de-4440-9ae4-d3e000daab17 None admin] PATH_INFO: `/v3/services` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:35.586536 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9535d8-72de-4440-9ae4-d3e000daab17 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:02:35.587054 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9535d8-72de-4440-9ae4-d3e000daab17 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:35.588717 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9535d8-72de-4440-9ae4-d3e000daab17 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:35.596357 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 445/892] 10.222.0.42 () {56 vars in 1114 bytes} [Mon Jul 25 12:02:35 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:02:35.599910 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188f942e-2719-4b16-af6b-17324d8cedff None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:35.603872 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9535d8-72de-4440-9ae4-d3e000daab17 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:35.649222 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c066298a-e3df-4d60-a443-0e2121e4ffea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:35.649915 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c066298a-e3df-4d60-a443-0e2121e4ffea None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:35.650208 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c066298a-e3df-4d60-a443-0e2121e4ffea None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:35.650491 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c066298a-e3df-4d60-a443-0e2121e4ffea None admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:35.651356 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c066298a-e3df-4d60-a443-0e2121e4ffea None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:02:35.651710 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c066298a-e3df-4d60-a443-0e2121e4ffea None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:35.653162 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c066298a-e3df-4d60-a443-0e2121e4ffea None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:35.677300 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 448/893] 10.222.0.42 () {56 vars in 1149 bytes} [Mon Jul 25 12:02:35 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 12:02:35.682290 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c066298a-e3df-4d60-a443-0e2121e4ffea None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:35.711133 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:35.722145 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9535d8-72de-4440-9ae4-d3e000daab17 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:35.758000 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55aff21-c84d-4a35-851b-0a935bae9a9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:35.759008 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55aff21-c84d-4a35-851b-0a935bae9a9c None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:35.759495 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55aff21-c84d-4a35-851b-0a935bae9a9c None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:35.760036 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55aff21-c84d-4a35-851b-0a935bae9a9c None admin] PATH_INFO: `/v3/services` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:35.760954 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55aff21-c84d-4a35-851b-0a935bae9a9c None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:02:35.761463 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55aff21-c84d-4a35-851b-0a935bae9a9c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:35.763018 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55aff21-c84d-4a35-851b-0a935bae9a9c None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:35.774172 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 446/894] 10.222.0.42 () {56 vars in 1114 bytes} [Mon Jul 25 12:02:35 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:02:35.775370 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55aff21-c84d-4a35-851b-0a935bae9a9c None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:35.814188 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c066298a-e3df-4d60-a443-0e2121e4ffea None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:35.862127 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a837572c-ce04-4cee-96c8-be6e8d1feda9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:35.862786 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a837572c-ce04-4cee-96c8-be6e8d1feda9 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:35.863076 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a837572c-ce04-4cee-96c8-be6e8d1feda9 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:35.863358 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a837572c-ce04-4cee-96c8-be6e8d1feda9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:35.884359 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:35.895476 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55aff21-c84d-4a35-851b-0a935bae9a9c None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:35.933001 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643daed8-e958-4f50-9caa-a91197d94dff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:35.934102 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643daed8-e958-4f50-9caa-a91197d94dff None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:35.934641 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643daed8-e958-4f50-9caa-a91197d94dff None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:35.935121 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643daed8-e958-4f50-9caa-a91197d94dff None admin] PATH_INFO: `/v3/endpoints` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:35.936346 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643daed8-e958-4f50-9caa-a91197d94dff None admin] RBAC: Adding query filter params (interface=public, service_id=44a51ccfc57a40a6a869a1b6e902dc43, region_id=RegionOne) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:02:35.936864 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643daed8-e958-4f50-9caa-a91197d94dff None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:35.939131 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643daed8-e958-4f50-9caa-a91197d94dff None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:35.950146 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 447/895] 10.222.0.42 () {56 vars in 1251 bytes} [Mon Jul 25 12:02:35 2022] GET /identity/v3/endpoints?service_id=44a51ccfc57a40a6a869a1b6e902dc43&interface=public®ion_id=RegionOne => generated 194 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:02:35.950750 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a837572c-ce04-4cee-96c8-be6e8d1feda9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:35.951586 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643daed8-e958-4f50-9caa-a91197d94dff None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:35.952238 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a837572c-ce04-4cee-96c8-be6e8d1feda9 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:35.973834 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:36.082083 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 449/896] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Jul 25 12:02:35 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 25 12:02:36.083004 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643daed8-e958-4f50-9caa-a91197d94dff None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:36.091568 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a837572c-ce04-4cee-96c8-be6e8d1feda9 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:36.129329 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f6c880-ec62-4f7b-ad52-9189c20d2b8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:36.130011 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f6c880-ec62-4f7b-ad52-9189c20d2b8f None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:36.130305 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f6c880-ec62-4f7b-ad52-9189c20d2b8f None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:36.130593 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f6c880-ec62-4f7b-ad52-9189c20d2b8f None admin] PATH_INFO: `/v3/services` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:36.131743 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f6c880-ec62-4f7b-ad52-9189c20d2b8f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:36.133131 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f6c880-ec62-4f7b-ad52-9189c20d2b8f None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:36.142145 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 448/897] 10.222.0.42 () {56 vars in 1087 bytes} [Mon Jul 25 12:02:35 2022] GET /identity/v3/services => generated 1268 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:02:36.147954 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:36.202221 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a837572c-ce04-4cee-96c8-be6e8d1feda9 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:36.241506 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8c2921-7109-4f20-b872-14335553a046 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:36.242265 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8c2921-7109-4f20-b872-14335553a046 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:36.242552 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8c2921-7109-4f20-b872-14335553a046 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:36.242833 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8c2921-7109-4f20-b872-14335553a046 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:36.254034 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8c2921-7109-4f20-b872-14335553a046 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:36.255514 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8c2921-7109-4f20-b872-14335553a046 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:36.275421 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-dc8c2921-7109-4f20-b872-14335553a046 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:02:36.275421 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:02:36.275421 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:02:36.275421 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:02:36.275421 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:02:36.275421 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:02:36.275421 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:02:36.275421 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:02:36.275421 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:02:36.275421 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:02:36.275421 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:02:36.275421 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:02:36.275421 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 12:02:36.275421 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 12:02:36.275421 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 12:02:36.276489 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 12:02:36.276489 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:02:36.276489 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:02:36.276489 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:02:36.276489 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:02:36.276489 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:02:36.276489 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:02:36.276489 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:02:36.276489 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 25 12:02:36.276489 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:02:36.276489 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:02:36.276489 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:02:36.276489 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:02:36.276489 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:02:36.276489 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:02:36.276489 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:02:36.276489 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:02:36.277364 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 12:02:36.277364 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 12:02:36.277364 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:02:36.277364 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 25 12:02:36.277364 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 450/898] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jul 25 12:02:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:02:36.283269 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f6c880-ec62-4f7b-ad52-9189c20d2b8f None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:36.375570 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:36.395477 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f6c880-ec62-4f7b-ad52-9189c20d2b8f None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:36.430785 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08013ee-9081-4576-a3b6-7d19295163bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:36.431461 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08013ee-9081-4576-a3b6-7d19295163bd None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:36.431784 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08013ee-9081-4576-a3b6-7d19295163bd None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:36.432075 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08013ee-9081-4576-a3b6-7d19295163bd None admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:36.432812 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08013ee-9081-4576-a3b6-7d19295163bd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:02:36.433130 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08013ee-9081-4576-a3b6-7d19295163bd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:36.434514 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08013ee-9081-4576-a3b6-7d19295163bd None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:36.447839 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 449/899] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jul 25 12:02:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:02:36.452976 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8c2921-7109-4f20-b872-14335553a046 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:36.530508 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191e657b-80d6-48b2-bdff-4a3fff27106b None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:36.530810 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191e657b-80d6-48b2-bdff-4a3fff27106b None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:36.531001 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191e657b-80d6-48b2-bdff-4a3fff27106b None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:36.533281 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 450/900] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 25 12:02:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:02:36.541159 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff5f9a5-5441-4bdb-a45d-2fcfd6083ebc None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:36.541460 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff5f9a5-5441-4bdb-a45d-2fcfd6083ebc None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:36.541745 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff5f9a5-5441-4bdb-a45d-2fcfd6083ebc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:36.570083 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-5ff5f9a5-5441-4bdb-a45d-2fcfd6083ebc None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:02:36.572188 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-5ff5f9a5-5441-4bdb-a45d-2fcfd6083ebc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:02:36.577986 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:36.584570 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8c2921-7109-4f20-b872-14335553a046 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:36.627167 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cf9f71-791e-437a-ba19-c93b23bd0fa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:36.628035 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cf9f71-791e-437a-ba19-c93b23bd0fa8 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:36.628327 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cf9f71-791e-437a-ba19-c93b23bd0fa8 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:36.628649 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cf9f71-791e-437a-ba19-c93b23bd0fa8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:36.637454 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49cf9f71-791e-437a-ba19-c93b23bd0fa8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:36.638905 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49cf9f71-791e-437a-ba19-c93b23bd0fa8 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:36.655787 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-49cf9f71-791e-437a-ba19-c93b23bd0fa8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 12:02:36.655787 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:02:36.655787 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:02:36.655787 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:02:36.655787 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:02:36.655787 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:02:36.655787 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:02:36.655787 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:02:36.655787 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:02:36.655787 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:02:36.655787 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:02:36.655787 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:02:36.655787 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 12:02:36.655787 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 12:02:36.655787 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 12:02:36.656690 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 12:02:36.656690 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:02:36.656690 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:02:36.656690 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:02:36.656690 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:02:36.656690 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:02:36.656690 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:02:36.656690 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:02:36.656690 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 25 12:02:36.656690 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:02:36.656690 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:02:36.656690 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:02:36.656690 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:02:36.656690 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:02:36.656690 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:02:36.656690 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:02:36.656690 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:02:36.657825 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 12:02:36.657825 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 12:02:36.657825 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 12:02:36.657825 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 12:02:36.657825 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 12:02:36.657825 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 12:02:36.657825 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 12:02:36.657825 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 25 12:02:36.657825 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 451/901] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Jul 25 12:02:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:02:36.662593 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cf9f71-791e-437a-ba19-c93b23bd0fa8 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:36.665899 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 451/902] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 25 12:02:36 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 12:02:36.689151 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff5f9a5-5441-4bdb-a45d-2fcfd6083ebc None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:36.775736 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:36.783792 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cf9f71-791e-437a-ba19-c93b23bd0fa8 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:36.818405 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff5f9a5-5441-4bdb-a45d-2fcfd6083ebc None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:36.820994 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce388f84-6ea2-460d-8307-2eb8ecb70ba3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:36.821940 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce388f84-6ea2-460d-8307-2eb8ecb70ba3 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:36.822430 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce388f84-6ea2-460d-8307-2eb8ecb70ba3 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:36.822924 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce388f84-6ea2-460d-8307-2eb8ecb70ba3 None admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:36.824019 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce388f84-6ea2-460d-8307-2eb8ecb70ba3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:02:36.824536 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce388f84-6ea2-460d-8307-2eb8ecb70ba3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:36.826246 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce388f84-6ea2-460d-8307-2eb8ecb70ba3 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:36.845027 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 452/903] 10.222.0.42 () {56 vars in 1148 bytes} [Mon Jul 25 12:02:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:02:36.846207 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce388f84-6ea2-460d-8307-2eb8ecb70ba3 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:36.859224 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d52ce17-7bc7-4201-a088-348597668639 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:36.860868 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d52ce17-7bc7-4201-a088-348597668639 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:36.861164 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d52ce17-7bc7-4201-a088-348597668639 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:36.861449 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d52ce17-7bc7-4201-a088-348597668639 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:36.862123 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d52ce17-7bc7-4201-a088-348597668639 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:36.863799 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d52ce17-7bc7-4201-a088-348597668639 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:36.868313 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:36.881343 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-1d52ce17-7bc7-4201-a088-348597668639 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 25 12:02:36.881343 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:02:36.881343 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 12:02:36.881343 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 12:02:36.881343 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 12:02:36.881343 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 12:02:36.881343 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 12:02:36.881343 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 12:02:36.881343 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 25 12:02:36.881343 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jul 25 12:02:36.881343 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 12:02:36.881343 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jul 25 12:02:36.881343 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:02:36.881343 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:02:36.881343 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:02:36.881343 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:02:36.882331 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:02:36.882331 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:02:36.882331 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:02:36.882331 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:02:36.882331 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:02:36.882331 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:02:36.882331 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:02:36.882331 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 12:02:36.882331 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 12:02:36.882331 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 12:02:36.882331 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 12:02:36.882331 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:02:36.882331 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:02:36.882331 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:02:36.882331 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:02:36.882331 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:02:36.883149 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:02:36.883149 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:02:36.883149 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jul 25 12:02:36.883149 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:02:36.883149 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:02:36.883149 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:02:36.883149 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:02:36.883149 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:02:36.883149 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:02:36.883149 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:02:36.883149 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:02:36.883149 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 12:02:36.883149 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 12:02:36.883149 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 25 12:02:36.883149 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jul 25 12:02:36.890786 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 452/904] 10.222.0.42 () {56 vars in 1132 bytes} [Mon Jul 25 12:02:36 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:02:36.892413 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d52ce17-7bc7-4201-a088-348597668639 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:36.976422 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce388f84-6ea2-460d-8307-2eb8ecb70ba3 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:37.000898 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d52ce17-7bc7-4201-a088-348597668639 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:37.028432 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fa9944-e73c-4cd0-8cc6-be1d50d31eea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:37.029103 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fa9944-e73c-4cd0-8cc6-be1d50d31eea None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:37.029396 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fa9944-e73c-4cd0-8cc6-be1d50d31eea None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:37.030084 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fa9944-e73c-4cd0-8cc6-be1d50d31eea None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:37.031290 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fa9944-e73c-4cd0-8cc6-be1d50d31eea None admin] RBAC: Adding query filter params (role.id=3297f946c8f04c9f8a159c7e36bf41a5, scope.project.id=b9f62b8f37304b8a9a4bb54205347577, user.id=93207b858d294d049fa56d45810a40aa) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:02:37.031643 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fa9944-e73c-4cd0-8cc6-be1d50d31eea None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:37.034445 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fa9944-e73c-4cd0-8cc6-be1d50d31eea None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:37.042354 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 453/905] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Jul 25 12:02:36 2022] GET /identity/v3/role_assignments?user.id=93207b858d294d049fa56d45810a40aa&scope.project.id=b9f62b8f37304b8a9a4bb54205347577&role.id=3297f946c8f04c9f8a159c7e36bf41a5 => generated 259 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:02:37.043382 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86ae582-f8dd-4e77-a69c-5dc3a1e9df1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:37.044295 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86ae582-f8dd-4e77-a69c-5dc3a1e9df1c None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:37.044747 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86ae582-f8dd-4e77-a69c-5dc3a1e9df1c None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:37.045187 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86ae582-f8dd-4e77-a69c-5dc3a1e9df1c None admin] PATH_INFO: `/v3/services` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:37.046075 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86ae582-f8dd-4e77-a69c-5dc3a1e9df1c None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:02:37.046574 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86ae582-f8dd-4e77-a69c-5dc3a1e9df1c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:37.048220 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86ae582-f8dd-4e77-a69c-5dc3a1e9df1c None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:37.050761 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:37.055936 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 453/906] 10.222.0.42 () {56 vars in 1126 bytes} [Mon Jul 25 12:02:36 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:02:37.061603 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fa9944-e73c-4cd0-8cc6-be1d50d31eea None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:37.155780 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:37.177013 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fa9944-e73c-4cd0-8cc6-be1d50d31eea None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:37.214824 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6042f685-46d3-422d-99db-5fa4bb9707aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:37.215626 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6042f685-46d3-422d-99db-5fa4bb9707aa None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:37.215813 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6042f685-46d3-422d-99db-5fa4bb9707aa None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:37.216098 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6042f685-46d3-422d-99db-5fa4bb9707aa None admin] PATH_INFO: `/v3/services` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:37.216815 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6042f685-46d3-422d-99db-5fa4bb9707aa None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:02:37.217126 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6042f685-46d3-422d-99db-5fa4bb9707aa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:37.218498 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6042f685-46d3-422d-99db-5fa4bb9707aa None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:37.224331 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 454/907] 10.222.0.42 () {56 vars in 1126 bytes} [Mon Jul 25 12:02:37 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:02:37.232775 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86ae582-f8dd-4e77-a69c-5dc3a1e9df1c None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:37.349303 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86ae582-f8dd-4e77-a69c-5dc3a1e9df1c None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:37.385880 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f02a53-0199-44d9-85f7-cdb2a7da1884 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:37.386548 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f02a53-0199-44d9-85f7-cdb2a7da1884 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:37.386837 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f02a53-0199-44d9-85f7-cdb2a7da1884 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:37.387121 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f02a53-0199-44d9-85f7-cdb2a7da1884 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:37.387965 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f02a53-0199-44d9-85f7-cdb2a7da1884 None admin] RBAC: Adding query filter params (interface=public, service_id=d0bc3b3b673f4f4ba17752345bb02f22, region_id=RegionOne) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:02:37.388284 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f02a53-0199-44d9-85f7-cdb2a7da1884 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:37.389652 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f02a53-0199-44d9-85f7-cdb2a7da1884 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:37.395378 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 454/908] 10.222.0.42 () {56 vars in 1251 bytes} [Mon Jul 25 12:02:37 2022] GET /identity/v3/endpoints?service_id=d0bc3b3b673f4f4ba17752345bb02f22&interface=public®ion_id=RegionOne => generated 194 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:02:37.400650 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6042f685-46d3-422d-99db-5fa4bb9707aa None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:37.421190 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3aec990-ebca-4e2f-8464-0d88bc7f640d None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:37.421526 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3aec990-ebca-4e2f-8464-0d88bc7f640d None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:37.422073 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3aec990-ebca-4e2f-8464-0d88bc7f640d None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:37.424381 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 455/909] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 25 12:02:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 12:02:37.432807 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b994831b-7381-4736-b364-d6b7fa7d0dc4 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:37.433106 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b994831b-7381-4736-b364-d6b7fa7d0dc4 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:37.433395 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b994831b-7381-4736-b364-d6b7fa7d0dc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:37.451183 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-b994831b-7381-4736-b364-d6b7fa7d0dc4 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:02:37.452434 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-b994831b-7381-4736-b364-d6b7fa7d0dc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:02:37.524235 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6042f685-46d3-422d-99db-5fa4bb9707aa None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:37.530678 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:37.548102 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 456/910] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 25 12:02:37 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 12:02:37.568365 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46418ee4-2d45-49e0-afc5-5df9bc4f8387 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:37.569292 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46418ee4-2d45-49e0-afc5-5df9bc4f8387 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:37.569762 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46418ee4-2d45-49e0-afc5-5df9bc4f8387 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:37.570259 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46418ee4-2d45-49e0-afc5-5df9bc4f8387 None admin] PATH_INFO: `/v3/services` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:37.571366 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b994831b-7381-4736-b364-d6b7fa7d0dc4 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:37.575197 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46418ee4-2d45-49e0-afc5-5df9bc4f8387 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:37.580732 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46418ee4-2d45-49e0-afc5-5df9bc4f8387 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:37.591638 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 455/911] 10.222.0.42 () {56 vars in 1087 bytes} [Mon Jul 25 12:02:37 2022] GET /identity/v3/services => generated 1268 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:02:37.699847 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b994831b-7381-4736-b364-d6b7fa7d0dc4 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:37.739355 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab9b93f-11ef-4bac-a5fe-55667a355072 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:37.740357 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab9b93f-11ef-4bac-a5fe-55667a355072 None admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:37.740791 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab9b93f-11ef-4bac-a5fe-55667a355072 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:37.741228 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab9b93f-11ef-4bac-a5fe-55667a355072 None admin] PATH_INFO: `/v3/services` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:37.742146 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab9b93f-11ef-4bac-a5fe-55667a355072 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:37.743747 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab9b93f-11ef-4bac-a5fe-55667a355072 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:37.767730 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 457/912] 10.222.0.42 () {60 vars in 1141 bytes} [Mon Jul 25 12:02:37 2022] POST /identity/v3/services => generated 241 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:02:38.247995 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d2e198-ec60-490f-b9b5-da6af20c5de6 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:38.248506 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d2e198-ec60-490f-b9b5-da6af20c5de6 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:38.248584 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d2e198-ec60-490f-b9b5-da6af20c5de6 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:38.250106 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 456/913] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 25 12:02:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:02:38.255459 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca38bf5-6a49-4d01-a8cb-2fe85b223a4b None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:38.255785 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca38bf5-6a49-4d01-a8cb-2fe85b223a4b None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:38.256068 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca38bf5-6a49-4d01-a8cb-2fe85b223a4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:38.276159 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-4ca38bf5-6a49-4d01-a8cb-2fe85b223a4b None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:02:38.277350 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-4ca38bf5-6a49-4d01-a8cb-2fe85b223a4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:02:38.329691 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:38.354575 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 458/914] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 25 12:02:38 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 12:02:38.376892 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d2e198-ec60-490f-b9b5-da6af20c5de6 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:38.389530 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:38.489114 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d2e198-ec60-490f-b9b5-da6af20c5de6 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:38.525277 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2592a650-499a-4e16-b80a-fc98cb1bca95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:38.526376 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2592a650-499a-4e16-b80a-fc98cb1bca95 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:38.526812 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2592a650-499a-4e16-b80a-fc98cb1bca95 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:38.527238 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2592a650-499a-4e16-b80a-fc98cb1bca95 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:38.528073 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2592a650-499a-4e16-b80a-fc98cb1bca95 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:38.529714 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2592a650-499a-4e16-b80a-fc98cb1bca95 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:38.542348 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-2592a650-499a-4e16-b80a-fc98cb1bca95 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 25 12:02:38.542348 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:02:38.542348 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 12:02:38.542348 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 12:02:38.542348 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 12:02:38.542348 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 12:02:38.542348 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 12:02:38.542348 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 12:02:38.542348 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 25 12:02:38.542348 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 25 12:02:38.542348 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 12:02:38.542348 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 25 12:02:38.542348 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:02:38.542348 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:02:38.542348 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:02:38.542348 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:02:38.542348 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:02:38.543588 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:02:38.543588 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:02:38.543588 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:02:38.543588 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:02:38.543588 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:02:38.543588 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:02:38.543588 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 12:02:38.543588 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 12:02:38.543588 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 12:02:38.543588 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 12:02:38.543588 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:02:38.543588 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:02:38.543588 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:02:38.543588 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:02:38.543588 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:02:38.543588 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:02:38.544699 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:02:38.544699 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 25 12:02:38.544699 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:02:38.544699 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:02:38.544699 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:02:38.544699 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:02:38.544699 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:02:38.544699 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:02:38.544699 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:02:38.544699 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:02:38.544699 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 12:02:38.544699 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 12:02:38.544699 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 25 12:02:38.544699 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 25 12:02:38.547147 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 457/915] 10.222.0.42 () {56 vars in 1114 bytes} [Mon Jul 25 12:02:38 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:02:38.556035 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca38bf5-6a49-4d01-a8cb-2fe85b223a4b None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:38.654180 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca38bf5-6a49-4d01-a8cb-2fe85b223a4b None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:38.690617 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f42a2fe-efcd-449c-a01e-ff5b6cc9eecb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:38.691288 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f42a2fe-efcd-449c-a01e-ff5b6cc9eecb None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:38.691605 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f42a2fe-efcd-449c-a01e-ff5b6cc9eecb None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:38.691894 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f42a2fe-efcd-449c-a01e-ff5b6cc9eecb None admin] PATH_INFO: `/v3/services` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:38.692604 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f42a2fe-efcd-449c-a01e-ff5b6cc9eecb None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:02:38.692915 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f42a2fe-efcd-449c-a01e-ff5b6cc9eecb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:38.694278 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f42a2fe-efcd-449c-a01e-ff5b6cc9eecb None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:38.699343 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 459/916] 10.222.0.42 () {56 vars in 1114 bytes} [Mon Jul 25 12:02:38 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:02:38.705576 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2592a650-499a-4e16-b80a-fc98cb1bca95 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:38.791916 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:38.816234 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2592a650-499a-4e16-b80a-fc98cb1bca95 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:38.851432 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16730284-e03c-481c-96d7-1dcfa3439714 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:38.852152 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16730284-e03c-481c-96d7-1dcfa3439714 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:38.852572 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16730284-e03c-481c-96d7-1dcfa3439714 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:38.852859 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16730284-e03c-481c-96d7-1dcfa3439714 None admin] PATH_INFO: `/v3/services` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:38.853587 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16730284-e03c-481c-96d7-1dcfa3439714 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:02:38.853902 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16730284-e03c-481c-96d7-1dcfa3439714 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:38.855337 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16730284-e03c-481c-96d7-1dcfa3439714 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:38.861023 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 458/917] 10.222.0.42 () {56 vars in 1114 bytes} [Mon Jul 25 12:02:38 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:02:38.866365 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f42a2fe-efcd-449c-a01e-ff5b6cc9eecb None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:38.912980 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:38.974202 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f42a2fe-efcd-449c-a01e-ff5b6cc9eecb None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:39.013489 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2395d43-3e07-4068-8b72-b2aa1c05a9fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:39.014176 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2395d43-3e07-4068-8b72-b2aa1c05a9fa None admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:39.014469 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2395d43-3e07-4068-8b72-b2aa1c05a9fa None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:39.014758 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2395d43-3e07-4068-8b72-b2aa1c05a9fa None admin] PATH_INFO: `/v3/endpoints` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:39.015561 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2395d43-3e07-4068-8b72-b2aa1c05a9fa None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:39.017560 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2395d43-3e07-4068-8b72-b2aa1c05a9fa None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:39.041398 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 460/918] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Jul 25 12:02:38 2022] POST /identity/v3/endpoints => generated 346 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:02:39.119681 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a31eee-13b8-4724-b323-ad5a1f167182 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:39.119976 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a31eee-13b8-4724-b323-ad5a1f167182 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:39.120260 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a31eee-13b8-4724-b323-ad5a1f167182 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:39.121947 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 459/919] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 25 12:02:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:02:39.127745 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13544975-e996-4e23-80c7-b5e636f0a7c6 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:39.128044 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13544975-e996-4e23-80c7-b5e636f0a7c6 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:39.128319 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13544975-e996-4e23-80c7-b5e636f0a7c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:39.148348 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-13544975-e996-4e23-80c7-b5e636f0a7c6 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:02:39.149575 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-13544975-e996-4e23-80c7-b5e636f0a7c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:02:39.234341 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 461/920] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 25 12:02:39 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 12:02:39.256408 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a31eee-13b8-4724-b323-ad5a1f167182 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:39.361915 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a31eee-13b8-4724-b323-ad5a1f167182 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:39.397094 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46e7202-98d6-4f43-b7c7-a90dc0935f17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:39.398159 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46e7202-98d6-4f43-b7c7-a90dc0935f17 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:39.398670 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46e7202-98d6-4f43-b7c7-a90dc0935f17 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:39.399141 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46e7202-98d6-4f43-b7c7-a90dc0935f17 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:39.406668 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:39.411826 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46e7202-98d6-4f43-b7c7-a90dc0935f17 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 12:02:39.420593 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-a46e7202-98d6-4f43-b7c7-a90dc0935f17 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 25 12:02:39.420593 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:02:39.420593 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:02:39.420593 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:02:39.420593 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:02:39.420593 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:02:39.420593 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:02:39.420593 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:02:39.420593 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:02:39.420593 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:02:39.420593 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:02:39.420593 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:02:39.420593 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 12:02:39.420593 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 12:02:39.420593 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 12:02:39.420593 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 12:02:39.421679 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 12:02:39.421679 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 12:02:39.421679 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 12:02:39.421679 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 12:02:39.421679 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:02:39.421679 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:02:39.421679 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:02:39.421679 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:02:39.421679 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:02:39.421679 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:02:39.421679 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:02:39.421679 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 25 12:02:39.421679 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:02:39.421679 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:02:39.421679 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:02:39.421679 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:02:39.422727 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:02:39.422727 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:02:39.422727 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:02:39.422727 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:02:39.422727 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 12:02:39.422727 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 12:02:39.422727 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 12:02:39.422727 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 12:02:39.422727 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 12:02:39.422727 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 12:02:39.422727 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 25 12:02:39.422727 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 25 12:02:39.429221 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 460/921] 10.222.0.42 () {56 vars in 1120 bytes} [Mon Jul 25 12:02:39 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:02:39.434943 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13544975-e996-4e23-80c7-b5e636f0a7c6 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:39.464310 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:39.539991 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13544975-e996-4e23-80c7-b5e636f0a7c6 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:39.576529 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8c476e-904f-4c29-87d5-a43ea0c41fc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:39.577617 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8c476e-904f-4c29-87d5-a43ea0c41fc7 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:39.578106 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8c476e-904f-4c29-87d5-a43ea0c41fc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:39.578636 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8c476e-904f-4c29-87d5-a43ea0c41fc7 None admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:39.579618 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8c476e-904f-4c29-87d5-a43ea0c41fc7 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:02:39.580131 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8c476e-904f-4c29-87d5-a43ea0c41fc7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:39.581726 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8c476e-904f-4c29-87d5-a43ea0c41fc7 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:39.590734 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 462/922] 10.222.0.42 () {56 vars in 1115 bytes} [Mon Jul 25 12:02:39 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:02:39.597121 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46e7202-98d6-4f43-b7c7-a90dc0935f17 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:39.704527 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46e7202-98d6-4f43-b7c7-a90dc0935f17 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:39.705948 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e33d91-8561-4e9c-a7be-c8426b72bd80 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:39.706249 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e33d91-8561-4e9c-a7be-c8426b72bd80 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:39.706537 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e33d91-8561-4e9c-a7be-c8426b72bd80 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:39.708223 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 463/923] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 25 12:02:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 12:02:39.713007 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbbd1e8-a733-42bd-acb0-07f23595b435 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:39.713304 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbbd1e8-a733-42bd-acb0-07f23595b435 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:39.713583 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbbd1e8-a733-42bd-acb0-07f23595b435 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:39.731778 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-dcbbd1e8-a733-42bd-acb0-07f23595b435 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:02:39.732987 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-dcbbd1e8-a733-42bd-acb0-07f23595b435 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:02:39.743390 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa9f350-fa87-4bd0-9786-b203acc1ec23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:39.744847 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa9f350-fa87-4bd0-9786-b203acc1ec23 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:39.745747 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa9f350-fa87-4bd0-9786-b203acc1ec23 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:39.746274 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa9f350-fa87-4bd0-9786-b203acc1ec23 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:39.760155 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa9f350-fa87-4bd0-9786-b203acc1ec23 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:39.761618 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa9f350-fa87-4bd0-9786-b203acc1ec23 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:39.789270 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-afa9f350-fa87-4bd0-9786-b203acc1ec23 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:02:39.789270 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:02:39.789270 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:02:39.789270 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:02:39.789270 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:02:39.789270 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:02:39.789270 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:02:39.789270 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:02:39.789270 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:02:39.789270 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:02:39.789270 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:02:39.789270 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:02:39.789270 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 12:02:39.789270 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 12:02:39.789270 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 12:02:39.790227 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 12:02:39.790227 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:02:39.790227 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:02:39.790227 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:02:39.790227 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:02:39.790227 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:02:39.790227 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:02:39.790227 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:02:39.790227 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 25 12:02:39.790227 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:02:39.790227 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:02:39.790227 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:02:39.790227 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:02:39.790227 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:02:39.790227 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:02:39.790227 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:02:39.790227 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:02:39.791385 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 12:02:39.791385 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 12:02:39.791385 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:02:39.791385 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 25 12:02:39.795487 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 461/924] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jul 25 12:02:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:02:39.796373 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa9f350-fa87-4bd0-9786-b203acc1ec23 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:39.823049 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 464/925] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 25 12:02:39 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 25 12:02:39.845018 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbbd1e8-a733-42bd-acb0-07f23595b435 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:39.861709 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:39.870015 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:39.930321 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa9f350-fa87-4bd0-9786-b203acc1ec23 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:39.962042 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbbd1e8-a733-42bd-acb0-07f23595b435 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:39.973812 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2905e16-8c0f-4738-a58e-2b972fd688e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:39.974923 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2905e16-8c0f-4738-a58e-2b972fd688e2 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:39.975311 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2905e16-8c0f-4738-a58e-2b972fd688e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:39.975727 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2905e16-8c0f-4738-a58e-2b972fd688e2 None admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:39.976696 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2905e16-8c0f-4738-a58e-2b972fd688e2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:02:39.977386 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2905e16-8c0f-4738-a58e-2b972fd688e2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:39.979055 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2905e16-8c0f-4738-a58e-2b972fd688e2 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:40.000314 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 462/926] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jul 25 12:02:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:02:40.002134 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2905e16-8c0f-4738-a58e-2b972fd688e2 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:40.004029 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60dbb0af-eaea-4682-a0f9-3ae067651422 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:40.004857 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dbb0af-eaea-4682-a0f9-3ae067651422 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:40.005147 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dbb0af-eaea-4682-a0f9-3ae067651422 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:40.005430 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dbb0af-eaea-4682-a0f9-3ae067651422 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:40.006121 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60dbb0af-eaea-4682-a0f9-3ae067651422 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:40.008090 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60dbb0af-eaea-4682-a0f9-3ae067651422 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:40.020843 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-60dbb0af-eaea-4682-a0f9-3ae067651422 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 25 12:02:40.020843 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:02:40.020843 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 12:02:40.020843 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 12:02:40.020843 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 12:02:40.020843 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 12:02:40.020843 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 12:02:40.020843 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 12:02:40.020843 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 25 12:02:40.020843 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jul 25 12:02:40.020843 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 12:02:40.020843 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jul 25 12:02:40.020843 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:02:40.020843 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:02:40.020843 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:02:40.020843 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:02:40.022011 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:02:40.022011 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:02:40.022011 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:02:40.022011 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:02:40.022011 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:02:40.022011 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:02:40.022011 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:02:40.022011 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 12:02:40.022011 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 12:02:40.022011 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 12:02:40.022011 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 12:02:40.022011 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:02:40.022011 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:02:40.022011 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:02:40.022011 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:02:40.022011 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:02:40.023035 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:02:40.023035 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:02:40.023035 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jul 25 12:02:40.023035 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:02:40.023035 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:02:40.023035 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:02:40.023035 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:02:40.023035 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:02:40.023035 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:02:40.023035 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:02:40.023035 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:02:40.023035 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 12:02:40.023035 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 12:02:40.023035 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 25 12:02:40.023035 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jul 25 12:02:40.030276 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 465/927] 10.222.0.42 () {56 vars in 1132 bytes} [Mon Jul 25 12:02:39 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 12:02:40.031338 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60dbb0af-eaea-4682-a0f9-3ae067651422 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:40.122175 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:40.126122 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2905e16-8c0f-4738-a58e-2b972fd688e2 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:40.137631 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:40.144107 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60dbb0af-eaea-4682-a0f9-3ae067651422 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:40.163868 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8453fea9-e566-437d-957b-4b5b3a558535 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:40.164679 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8453fea9-e566-437d-957b-4b5b3a558535 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:40.165385 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8453fea9-e566-437d-957b-4b5b3a558535 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:40.165926 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8453fea9-e566-437d-957b-4b5b3a558535 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:40.173822 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8453fea9-e566-437d-957b-4b5b3a558535 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:40.175277 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8453fea9-e566-437d-957b-4b5b3a558535 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:40.180082 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83814b5-7e92-4853-bc0c-6300ae0f38ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:40.180762 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83814b5-7e92-4853-bc0c-6300ae0f38ad None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:40.181052 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83814b5-7e92-4853-bc0c-6300ae0f38ad None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:40.181360 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83814b5-7e92-4853-bc0c-6300ae0f38ad None admin] PATH_INFO: `/v3/services` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:40.182078 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83814b5-7e92-4853-bc0c-6300ae0f38ad None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:02:40.182391 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83814b5-7e92-4853-bc0c-6300ae0f38ad None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:40.183786 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83814b5-7e92-4853-bc0c-6300ae0f38ad None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:40.193352 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 466/928] 10.222.0.42 () {56 vars in 1126 bytes} [Mon Jul 25 12:02:40 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:02:40.193352 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bb5ef5-94cd-40a3-b6b6-c5873804e768 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:40.193352 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bb5ef5-94cd-40a3-b6b6-c5873804e768 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:40.193615 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-8453fea9-e566-437d-957b-4b5b3a558535 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 25 12:02:40.193615 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:02:40.193615 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:02:40.193615 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:02:40.193615 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:02:40.193615 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:02:40.193615 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:02:40.193615 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:02:40.193615 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:02:40.193615 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:02:40.193615 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:02:40.193615 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:02:40.193615 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 12:02:40.193615 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 12:02:40.193615 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 12:02:40.193615 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 12:02:40.194516 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:02:40.194516 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:02:40.194516 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 12:02:40.194516 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 12:02:40.194516 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 12:02:40.194516 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 12:02:40.194516 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:02:40.194516 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:02:40.194516 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:02:40.194516 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:02:40.194516 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:02:40.194516 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 25 12:02:40.194516 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:02:40.194516 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:02:40.194516 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:02:40.194516 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:02:40.194516 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:02:40.194516 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:02:40.195453 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:02:40.195453 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:02:40.195453 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 12:02:40.195453 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 12:02:40.195453 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 12:02:40.195453 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 12:02:40.195453 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 12:02:40.195453 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 12:02:40.195453 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 25 12:02:40.195453 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 25 12:02:40.196034 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bb5ef5-94cd-40a3-b6b6-c5873804e768 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:40.196104 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 463/929] 10.222.0.42 () {56 vars in 1117 bytes} [Mon Jul 25 12:02:40 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:02:40.196170 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 467/930] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 25 12:02:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:02:40.196236 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8453fea9-e566-437d-957b-4b5b3a558535 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:40.196917 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bb5ef5-94cd-40a3-b6b6-c5873804e768 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:40.317447 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bb5ef5-94cd-40a3-b6b6-c5873804e768 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:40.326616 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8453fea9-e566-437d-957b-4b5b3a558535 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:40.353447 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00da6700-9857-4ede-80c0-6dbc8c7cdc5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:40.354125 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00da6700-9857-4ede-80c0-6dbc8c7cdc5a None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:40.354442 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00da6700-9857-4ede-80c0-6dbc8c7cdc5a None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:40.354728 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00da6700-9857-4ede-80c0-6dbc8c7cdc5a None admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:40.355595 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00da6700-9857-4ede-80c0-6dbc8c7cdc5a None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:02:40.355921 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00da6700-9857-4ede-80c0-6dbc8c7cdc5a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:40.357356 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00da6700-9857-4ede-80c0-6dbc8c7cdc5a None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:40.367130 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd441d4-2010-4b69-b151-871e8763334c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:40.367932 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd441d4-2010-4b69-b151-871e8763334c None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:40.368223 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd441d4-2010-4b69-b151-871e8763334c None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:40.368507 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd441d4-2010-4b69-b151-871e8763334c None admin] PATH_INFO: `/v3/services` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:40.369224 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd441d4-2010-4b69-b151-871e8763334c None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:02:40.369540 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd441d4-2010-4b69-b151-871e8763334c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:40.370894 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd441d4-2010-4b69-b151-871e8763334c None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:40.375655 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 468/931] 10.222.0.42 () {56 vars in 1149 bytes} [Mon Jul 25 12:02:40 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 12:02:40.376871 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6399df68-0764-4e0e-b755-9188b5246554 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:40.377168 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6399df68-0764-4e0e-b755-9188b5246554 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:40.377453 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6399df68-0764-4e0e-b755-9188b5246554 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:40.385673 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 464/932] 10.222.0.42 () {56 vars in 1126 bytes} [Mon Jul 25 12:02:40 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:02:40.385673 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd441d4-2010-4b69-b151-871e8763334c None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:40.389321 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:40.400458 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-6399df68-0764-4e0e-b755-9188b5246554 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:02:40.401656 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-6399df68-0764-4e0e-b755-9188b5246554 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:02:40.423959 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:40.472943 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 469/933] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 25 12:02:40 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 12:02:40.474395 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6399df68-0764-4e0e-b755-9188b5246554 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:40.513836 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd441d4-2010-4b69-b151-871e8763334c None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:40.553764 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d6fc83-1562-4b3c-b212-656f8a2284e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:40.555308 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d6fc83-1562-4b3c-b212-656f8a2284e0 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:40.555816 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d6fc83-1562-4b3c-b212-656f8a2284e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:40.556275 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d6fc83-1562-4b3c-b212-656f8a2284e0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:40.568534 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d6fc83-1562-4b3c-b212-656f8a2284e0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:40.570216 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d6fc83-1562-4b3c-b212-656f8a2284e0 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:40.580634 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6399df68-0764-4e0e-b755-9188b5246554 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:40.588867 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-64d6fc83-1562-4b3c-b212-656f8a2284e0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:02:40.588867 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:02:40.588867 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:02:40.588867 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:02:40.588867 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:02:40.588867 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:02:40.588867 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:02:40.588867 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:02:40.588867 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:02:40.588867 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:02:40.588867 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:02:40.588867 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:02:40.588867 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 12:02:40.588867 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 12:02:40.588867 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 12:02:40.589769 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 12:02:40.589769 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:02:40.589769 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:02:40.589769 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:02:40.589769 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:02:40.589769 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:02:40.589769 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:02:40.589769 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:02:40.589769 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 25 12:02:40.589769 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:02:40.589769 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:02:40.589769 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:02:40.589769 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:02:40.589769 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:02:40.589769 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:02:40.589769 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:02:40.589769 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:02:40.596072 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 12:02:40.596072 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 12:02:40.596072 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:02:40.596072 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 25 12:02:40.596072 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 465/934] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jul 25 12:02:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:02:40.596072 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d6fc83-1562-4b3c-b212-656f8a2284e0 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:40.627821 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b04f3ff-fbbd-4308-86e3-fff434d45134 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:40.628497 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b04f3ff-fbbd-4308-86e3-fff434d45134 None admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:40.628791 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b04f3ff-fbbd-4308-86e3-fff434d45134 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:40.629077 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b04f3ff-fbbd-4308-86e3-fff434d45134 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:40.629832 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b04f3ff-fbbd-4308-86e3-fff434d45134 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:40.631208 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b04f3ff-fbbd-4308-86e3-fff434d45134 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:40.636206 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:40.654264 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 470/935] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Jul 25 12:02:40 2022] POST /identity/v3/endpoints => generated 347 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:02:40.656592 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b04f3ff-fbbd-4308-86e3-fff434d45134 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:40.698704 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:40.723985 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d6fc83-1562-4b3c-b212-656f8a2284e0 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:40.771860 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3acad9-9f2f-44f9-b854-401bdd744baa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:40.773162 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3acad9-9f2f-44f9-b854-401bdd744baa None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:40.773635 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3acad9-9f2f-44f9-b854-401bdd744baa None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:40.774131 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3acad9-9f2f-44f9-b854-401bdd744baa None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:40.774997 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3acad9-9f2f-44f9-b854-401bdd744baa None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:40.776802 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3acad9-9f2f-44f9-b854-401bdd744baa None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:40.791920 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-ca3acad9-9f2f-44f9-b854-401bdd744baa None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 25 12:02:40.791920 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:02:40.791920 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 12:02:40.791920 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 12:02:40.791920 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 12:02:40.791920 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 12:02:40.791920 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 12:02:40.791920 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 12:02:40.791920 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 25 12:02:40.791920 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 25 12:02:40.791920 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 12:02:40.791920 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 25 12:02:40.791920 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:02:40.791920 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:02:40.791920 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:02:40.791920 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:02:40.791920 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:02:40.792918 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:02:40.792918 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:02:40.792918 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:02:40.792918 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:02:40.792918 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:02:40.792918 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:02:40.792918 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 12:02:40.792918 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 12:02:40.792918 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 12:02:40.792918 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 12:02:40.792918 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:02:40.792918 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:02:40.792918 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:02:40.792918 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:02:40.792918 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:02:40.792918 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:02:40.793785 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:02:40.793785 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 25 12:02:40.793785 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:02:40.793785 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:02:40.793785 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:02:40.793785 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:02:40.793785 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:02:40.793785 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:02:40.793785 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:02:40.793785 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:02:40.793785 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 12:02:40.793785 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 12:02:40.793785 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 25 12:02:40.793785 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 25 12:02:40.797086 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 466/936] 10.222.0.42 () {56 vars in 1126 bytes} [Mon Jul 25 12:02:40 2022] GET /identity/v3/services/object-store => generated 93 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:02:40.803275 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3acad9-9f2f-44f9-b854-401bdd744baa None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:40.814735 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b04f3ff-fbbd-4308-86e3-fff434d45134 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:40.868363 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d30bfe0-b7c9-4523-92f5-e2aea4897d91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:40.868794 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d30bfe0-b7c9-4523-92f5-e2aea4897d91 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:40.868936 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d30bfe0-b7c9-4523-92f5-e2aea4897d91 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:40.869171 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d30bfe0-b7c9-4523-92f5-e2aea4897d91 None admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:40.869904 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d30bfe0-b7c9-4523-92f5-e2aea4897d91 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:02:40.870218 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d30bfe0-b7c9-4523-92f5-e2aea4897d91 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:40.871750 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d30bfe0-b7c9-4523-92f5-e2aea4897d91 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:40.887012 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 471/937] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jul 25 12:02:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:02:40.892328 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d30bfe0-b7c9-4523-92f5-e2aea4897d91 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:40.935686 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3acad9-9f2f-44f9-b854-401bdd744baa None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:40.956250 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:40.976198 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bc8b97-4b29-4a9a-b9d5-92e35e7a3cac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:40.977662 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bc8b97-4b29-4a9a-b9d5-92e35e7a3cac None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:40.978426 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bc8b97-4b29-4a9a-b9d5-92e35e7a3cac None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:40.979142 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bc8b97-4b29-4a9a-b9d5-92e35e7a3cac None admin] PATH_INFO: `/v3/services` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:40.980791 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43bc8b97-4b29-4a9a-b9d5-92e35e7a3cac None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:02:40.981583 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43bc8b97-4b29-4a9a-b9d5-92e35e7a3cac None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:40.983269 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43bc8b97-4b29-4a9a-b9d5-92e35e7a3cac None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:40.990288 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 467/938] 10.222.0.42 () {56 vars in 1122 bytes} [Mon Jul 25 12:02:40 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:02:41.003592 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bc8b97-4b29-4a9a-b9d5-92e35e7a3cac None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:41.014460 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d30bfe0-b7c9-4523-92f5-e2aea4897d91 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:41.019070 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:41.066360 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7778b2f9-b8e0-4776-8cfe-4250a576fd64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:41.067781 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7778b2f9-b8e0-4776-8cfe-4250a576fd64 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:41.068223 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7778b2f9-b8e0-4776-8cfe-4250a576fd64 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:41.068652 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7778b2f9-b8e0-4776-8cfe-4250a576fd64 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:41.078615 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7778b2f9-b8e0-4776-8cfe-4250a576fd64 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:41.081088 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7778b2f9-b8e0-4776-8cfe-4250a576fd64 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:41.100467 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-7778b2f9-b8e0-4776-8cfe-4250a576fd64 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 12:02:41.100467 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:02:41.100467 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:02:41.100467 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:02:41.100467 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:02:41.100467 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:02:41.100467 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:02:41.100467 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:02:41.100467 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:02:41.100467 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:02:41.100467 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:02:41.100467 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:02:41.100467 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 12:02:41.100467 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 12:02:41.100467 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 12:02:41.101568 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 12:02:41.101568 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:02:41.101568 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:02:41.101568 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:02:41.101568 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:02:41.101568 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:02:41.101568 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:02:41.101568 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:02:41.101568 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jul 25 12:02:41.101568 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:02:41.101568 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:02:41.101568 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:02:41.101568 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:02:41.101568 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:02:41.101568 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:02:41.101568 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:02:41.101568 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:02:41.102673 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 12:02:41.102673 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 12:02:41.102673 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 12:02:41.102673 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 12:02:41.102673 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 12:02:41.102673 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 12:02:41.102673 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 12:02:41.102673 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jul 25 12:02:41.105315 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 472/939] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Jul 25 12:02:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 12:02:41.110859 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7778b2f9-b8e0-4776-8cfe-4250a576fd64 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:41.131689 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bc8b97-4b29-4a9a-b9d5-92e35e7a3cac None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:41.172998 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437d8726-5cb8-4668-b37b-a181a419f886 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:41.173930 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437d8726-5cb8-4668-b37b-a181a419f886 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:41.174637 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437d8726-5cb8-4668-b37b-a181a419f886 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:41.175525 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437d8726-5cb8-4668-b37b-a181a419f886 None admin] PATH_INFO: `/v3/services` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:41.176702 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437d8726-5cb8-4668-b37b-a181a419f886 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:02:41.177169 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437d8726-5cb8-4668-b37b-a181a419f886 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:41.179361 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437d8726-5cb8-4668-b37b-a181a419f886 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:41.186474 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 468/940] 10.222.0.42 () {56 vars in 1122 bytes} [Mon Jul 25 12:02:40 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:02:41.188484 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca94dfe-34a9-44f7-9be9-2dde6b174351 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:41.188926 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca94dfe-34a9-44f7-9be9-2dde6b174351 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:41.189355 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca94dfe-34a9-44f7-9be9-2dde6b174351 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:41.190990 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 469/941] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 25 12:02:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:02:41.193930 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca94dfe-34a9-44f7-9be9-2dde6b174351 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:41.236119 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7778b2f9-b8e0-4776-8cfe-4250a576fd64 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:41.249483 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:41.274715 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:41.280599 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b498c3-0121-43d8-8fc2-f226640b713e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:41.281640 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b498c3-0121-43d8-8fc2-f226640b713e None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:41.282138 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b498c3-0121-43d8-8fc2-f226640b713e None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:41.282652 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b498c3-0121-43d8-8fc2-f226640b713e None admin] PATH_INFO: `/v3/projects` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:41.284057 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b498c3-0121-43d8-8fc2-f226640b713e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:02:41.284824 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b498c3-0121-43d8-8fc2-f226640b713e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:41.286873 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b498c3-0121-43d8-8fc2-f226640b713e None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:41.303657 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 473/942] 10.222.0.42 () {56 vars in 1148 bytes} [Mon Jul 25 12:02:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 12:02:41.306057 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50954aa8-4dab-4bf0-a34c-289e26e3f5d4 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:41.306409 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50954aa8-4dab-4bf0-a34c-289e26e3f5d4 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:41.306773 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50954aa8-4dab-4bf0-a34c-289e26e3f5d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:41.311424 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca94dfe-34a9-44f7-9be9-2dde6b174351 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:41.330016 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-50954aa8-4dab-4bf0-a34c-289e26e3f5d4 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:02:41.331223 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-50954aa8-4dab-4bf0-a34c-289e26e3f5d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:02:41.351202 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec533f83-6866-45df-a23d-99245e7e3aeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:41.352540 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec533f83-6866-45df-a23d-99245e7e3aeb None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:41.353038 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec533f83-6866-45df-a23d-99245e7e3aeb None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:41.353997 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec533f83-6866-45df-a23d-99245e7e3aeb None admin] PATH_INFO: `/v3/endpoints` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:41.355409 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec533f83-6866-45df-a23d-99245e7e3aeb None admin] RBAC: Adding query filter params (interface=public, service_id=5b1edfdedd42440e93e3c22748838045, region_id=RegionOne) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:02:41.356136 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec533f83-6866-45df-a23d-99245e7e3aeb None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:41.358392 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec533f83-6866-45df-a23d-99245e7e3aeb None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:41.367952 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 470/943] 10.222.0.42 () {56 vars in 1251 bytes} [Mon Jul 25 12:02:41 2022] GET /identity/v3/endpoints?service_id=5b1edfdedd42440e93e3c22748838045&interface=public®ion_id=RegionOne => generated 194 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 12:02:41.368584 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec533f83-6866-45df-a23d-99245e7e3aeb None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:41.407399 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 474/944] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 25 12:02:41 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 12:02:41.408202 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50954aa8-4dab-4bf0-a34c-289e26e3f5d4 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:41.479650 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec533f83-6866-45df-a23d-99245e7e3aeb None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:41.518385 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:41.520419 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b78be7d-f185-4551-9d05-de7de6c90ca3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:41.521408 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50954aa8-4dab-4bf0-a34c-289e26e3f5d4 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:41.521924 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b78be7d-f185-4551-9d05-de7de6c90ca3 None admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:41.522357 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b78be7d-f185-4551-9d05-de7de6c90ca3 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:41.522782 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b78be7d-f185-4551-9d05-de7de6c90ca3 None admin] PATH_INFO: `/v3/projects/b9f62b8f37304b8a9a4bb54205347577/users/93207b858d294d049fa56d45810a40aa/roles/3297f946c8f04c9f8a159c7e36bf41a5` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:41.525901 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b78be7d-f185-4551-9d05-de7de6c90ca3 None admin] RBAC: Authorizing `identity:create_grant(project_id=b9f62b8f37304b8a9a4bb54205347577, user_id=93207b858d294d049fa56d45810a40aa, role_id=3297f946c8f04c9f8a159c7e36bf41a5)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:41.527485 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b78be7d-f185-4551-9d05-de7de6c90ca3 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:41.536492 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:41.543041 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 471/945] 10.222.0.42 () {58 vars in 1439 bytes} [Mon Jul 25 12:02:41 2022] PUT /identity/v3/projects/b9f62b8f37304b8a9a4bb54205347577/users/93207b858d294d049fa56d45810a40aa/roles/3297f946c8f04c9f8a159c7e36bf41a5 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:02:41.543985 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b78be7d-f185-4551-9d05-de7de6c90ca3 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:41.572279 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a0a187-c869-4189-9ed1-c3bf2e0afe72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:41.573567 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a0a187-c869-4189-9ed1-c3bf2e0afe72 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:41.574235 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a0a187-c869-4189-9ed1-c3bf2e0afe72 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:41.574672 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a0a187-c869-4189-9ed1-c3bf2e0afe72 None admin] PATH_INFO: `/v3/services` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:41.575524 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a0a187-c869-4189-9ed1-c3bf2e0afe72 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:41.577463 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a0a187-c869-4189-9ed1-c3bf2e0afe72 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:41.585082 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 475/946] 10.222.0.42 () {56 vars in 1087 bytes} [Mon Jul 25 12:02:41 2022] GET /identity/v3/services => generated 1497 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:02:41.718486 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b78be7d-f185-4551-9d05-de7de6c90ca3 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:41.757109 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7f719f-c9ad-461d-ac04-d0358d032c1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:41.757784 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7f719f-c9ad-461d-ac04-d0358d032c1f None admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:41.758066 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7f719f-c9ad-461d-ac04-d0358d032c1f None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:41.758342 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7f719f-c9ad-461d-ac04-d0358d032c1f None admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:41.759055 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7f719f-c9ad-461d-ac04-d0358d032c1f None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:41.760525 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7f719f-c9ad-461d-ac04-d0358d032c1f None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:41.763218 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: WARNING keystone.server.flask.common [None req-bc7f719f-c9ad-461d-ac04-d0358d032c1f None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jul 25 12:02:41.783174 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.sql.core [None req-bc7f719f-c9ad-461d-ac04-d0358d032c1f None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 25 12:02:41.783174 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 25 12:02:41.783174 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [parameters: {'id': 'c31416ff2fba46ad94e4acc01de3fcda', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 25 12:02:41.783174 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80297) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 25 12:02:41.800495 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-bc7f719f-c9ad-461d-ac04-d0358d032c1f None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id c31416ff2fba46ad94e4acc01de3fcda.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id c31416ff2fba46ad94e4acc01de3fcda. Jul 25 12:02:41.800495 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:02:41.800495 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 25 12:02:41.800495 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 25 12:02:41.800495 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 25 12:02:41.800495 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 25 12:02:41.800495 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 25 12:02:41.800495 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application result = self._query(query) Jul 25 12:02:41.800495 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 25 12:02:41.800495 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application conn.query(q) Jul 25 12:02:41.800495 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 25 12:02:41.800495 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 25 12:02:41.800495 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 25 12:02:41.801408 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application result.read() Jul 25 12:02:41.801408 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 25 12:02:41.801408 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 25 12:02:41.801408 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 25 12:02:41.801408 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 25 12:02:41.801408 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 25 12:02:41.801408 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 25 12:02:41.801408 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 25 12:02:41.801408 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 25 12:02:41.801408 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 25 12:02:41.801408 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 25 12:02:41.801408 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 25 12:02:41.801408 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 25 12:02:41.801408 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:02:41.801408 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 25 12:02:41.801408 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 25 12:02:41.801408 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jul 25 12:02:41.801408 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return project_ref.to_dict() Jul 25 12:02:41.802375 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 25 12:02:41.802375 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application next(self.gen) Jul 25 12:02:41.802375 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 25 12:02:41.802375 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application yield resource Jul 25 12:02:41.802375 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 25 12:02:41.802375 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application next(self.gen) Jul 25 12:02:41.802375 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 25 12:02:41.802375 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application self.session.rollback() Jul 25 12:02:41.802375 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 25 12:02:41.802375 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application self.force_reraise() Jul 25 12:02:41.802375 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 25 12:02:41.802375 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise self.value Jul 25 12:02:41.802375 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 25 12:02:41.802375 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 25 12:02:41.802375 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 25 12:02:41.802375 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application session.commit() Jul 25 12:02:41.802375 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 25 12:02:41.803440 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 25 12:02:41.803440 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 25 12:02:41.803440 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application self._prepare_impl() Jul 25 12:02:41.803440 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 25 12:02:41.803440 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application self.session.flush() Jul 25 12:02:41.803440 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 25 12:02:41.803440 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application self._flush(objects) Jul 25 12:02:41.803440 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 25 12:02:41.803440 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 25 12:02:41.803440 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 25 12:02:41.803440 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application compat.raise_( Jul 25 12:02:41.803440 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 25 12:02:41.803440 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception Jul 25 12:02:41.803440 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 25 12:02:41.803440 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application flush_context.execute() Jul 25 12:02:41.803440 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 25 12:02:41.803440 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rec.execute(self) Jul 25 12:02:41.804550 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 25 12:02:41.804550 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 25 12:02:41.804550 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 25 12:02:41.804550 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 25 12:02:41.804550 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 25 12:02:41.804550 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 25 12:02:41.804550 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 25 12:02:41.804550 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 25 12:02:41.804550 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 25 12:02:41.804550 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 25 12:02:41.804550 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 25 12:02:41.804550 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 25 12:02:41.804550 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 25 12:02:41.804550 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 25 12:02:41.804550 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 25 12:02:41.805418 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 25 12:02:41.805418 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 25 12:02:41.805418 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception Jul 25 12:02:41.805418 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 25 12:02:41.805418 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 25 12:02:41.805418 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 25 12:02:41.805418 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 25 12:02:41.805418 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 25 12:02:41.805418 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application result = self._query(query) Jul 25 12:02:41.805418 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 25 12:02:41.805418 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application conn.query(q) Jul 25 12:02:41.805418 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 25 12:02:41.805418 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 25 12:02:41.805418 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 25 12:02:41.805418 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application result.read() Jul 25 12:02:41.805418 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 25 12:02:41.805418 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 25 12:02:41.806311 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 25 12:02:41.806311 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 25 12:02:41.806311 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 25 12:02:41.806311 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 25 12:02:41.806311 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 25 12:02:41.806311 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 25 12:02:41.806311 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 25 12:02:41.806311 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 25 12:02:41.806311 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [parameters: {'id': 'c31416ff2fba46ad94e4acc01de3fcda', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 25 12:02:41.806311 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 25 12:02:41.806311 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 25 12:02:41.806311 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 12:02:41.806311 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 25 12:02:41.806311 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:02:41.806311 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jul 25 12:02:41.807107 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jul 25 12:02:41.807107 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 25 12:02:41.807107 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 25 12:02:41.807107 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jul 25 12:02:41.807107 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 25 12:02:41.807107 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 12:02:41.807107 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 25 12:02:41.807107 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:02:41.807107 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:02:41.807107 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:02:41.807107 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:02:41.807107 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:02:41.807107 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:02:41.807107 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:02:41.807107 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:02:41.807107 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:02:41.807107 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:02:41.808061 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:02:41.808061 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jul 25 12:02:41.808061 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jul 25 12:02:41.808061 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:02:41.808061 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:02:41.808061 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jul 25 12:02:41.808061 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.Conflict( Jul 25 12:02:41.808061 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id c31416ff2fba46ad94e4acc01de3fcda. Jul 25 12:02:41.808061 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 25 12:02:41.808061 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 472/947] 10.222.0.42 () {60 vars in 1140 bytes} [Mon Jul 25 12:02:41 2022] POST /identity/v3/projects => generated 262 bytes in 258 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:02:41.811855 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a0a187-c869-4189-9ed1-c3bf2e0afe72 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:41.919585 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a0a187-c869-4189-9ed1-c3bf2e0afe72 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:41.957149 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b0b5f8-6980-48a3-ba01-1951f125660d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:41.958127 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b0b5f8-6980-48a3-ba01-1951f125660d None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:41.958517 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b0b5f8-6980-48a3-ba01-1951f125660d None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:41.958899 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b0b5f8-6980-48a3-ba01-1951f125660d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:41.966134 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b0b5f8-6980-48a3-ba01-1951f125660d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:41.967727 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b0b5f8-6980-48a3-ba01-1951f125660d None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:41.981755 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-a1b0b5f8-6980-48a3-ba01-1951f125660d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 12:02:41.981755 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:02:41.981755 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:02:41.981755 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:02:41.981755 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:02:41.981755 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:02:41.981755 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:02:41.981755 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:02:41.981755 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:02:41.981755 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:02:41.981755 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:02:41.981755 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:02:41.981755 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 12:02:41.981755 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 12:02:41.981755 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 12:02:41.982818 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 12:02:41.982818 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:02:41.982818 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:02:41.982818 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:02:41.982818 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:02:41.982818 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:02:41.982818 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:02:41.982818 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:02:41.982818 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jul 25 12:02:41.982818 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:02:41.982818 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:02:41.982818 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:02:41.982818 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:02:41.982818 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:02:41.982818 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:02:41.982818 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:02:41.982818 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:02:41.983953 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 12:02:41.983953 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 12:02:41.983953 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 12:02:41.983953 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 12:02:41.983953 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 12:02:41.983953 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 12:02:41.983953 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 12:02:41.983953 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jul 25 12:02:41.990174 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 476/948] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Jul 25 12:02:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:02:41.992854 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7f719f-c9ad-461d-ac04-d0358d032c1f None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:42.054726 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:42.104046 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7f719f-c9ad-461d-ac04-d0358d032c1f None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:42.143059 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280c18cf-1761-41f9-8170-2cf99d01f3e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:42.143762 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280c18cf-1761-41f9-8170-2cf99d01f3e3 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:42.144053 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280c18cf-1761-41f9-8170-2cf99d01f3e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:42.144338 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280c18cf-1761-41f9-8170-2cf99d01f3e3 None admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:42.145113 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280c18cf-1761-41f9-8170-2cf99d01f3e3 None admin] RBAC: Adding query filter params (name=service) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:02:42.145425 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280c18cf-1761-41f9-8170-2cf99d01f3e3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:42.147077 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280c18cf-1761-41f9-8170-2cf99d01f3e3 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:42.168780 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 473/949] 10.222.0.42 () {56 vars in 1112 bytes} [Mon Jul 25 12:02:41 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:02:42.774606 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffee2f2-2eae-4793-8116-f9e63d6f1401 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:42.775126 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffee2f2-2eae-4793-8116-f9e63d6f1401 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:42.775212 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffee2f2-2eae-4793-8116-f9e63d6f1401 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:42.776853 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 477/950] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 25 12:02:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:02:42.782266 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbd1cc0-03fc-4b38-8aef-d90db4cef2cf None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:42.782693 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbd1cc0-03fc-4b38-8aef-d90db4cef2cf None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:42.782841 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbd1cc0-03fc-4b38-8aef-d90db4cef2cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:42.802711 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-dcbd1cc0-03fc-4b38-8aef-d90db4cef2cf None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:02:42.804301 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-dcbd1cc0-03fc-4b38-8aef-d90db4cef2cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:02:42.877123 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 474/951] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 25 12:02:42 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 12:02:42.899151 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffee2f2-2eae-4793-8116-f9e63d6f1401 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:42.925879 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:43.011805 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffee2f2-2eae-4793-8116-f9e63d6f1401 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:43.048884 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2feeab-0859-4e51-a184-5b978220b8f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:43.049683 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2feeab-0859-4e51-a184-5b978220b8f8 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:43.049971 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2feeab-0859-4e51-a184-5b978220b8f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:43.050252 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2feeab-0859-4e51-a184-5b978220b8f8 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:43.050915 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2feeab-0859-4e51-a184-5b978220b8f8 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:43.052312 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2feeab-0859-4e51-a184-5b978220b8f8 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:43.064251 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-2d2feeab-0859-4e51-a184-5b978220b8f8 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 25 12:02:43.064251 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:02:43.064251 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 12:02:43.064251 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 12:02:43.064251 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 12:02:43.064251 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 12:02:43.064251 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 12:02:43.064251 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 12:02:43.064251 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 25 12:02:43.064251 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jul 25 12:02:43.064251 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 12:02:43.064251 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jul 25 12:02:43.064251 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:02:43.064251 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:02:43.064251 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:02:43.064251 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:02:43.064251 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:02:43.065226 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:02:43.065226 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:02:43.065226 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:02:43.065226 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:02:43.065226 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:02:43.065226 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:02:43.065226 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 12:02:43.065226 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 12:02:43.065226 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 12:02:43.065226 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 12:02:43.065226 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:02:43.065226 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:02:43.065226 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:02:43.065226 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:02:43.065226 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:02:43.065226 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:02:43.066060 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:02:43.066060 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jul 25 12:02:43.066060 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:02:43.066060 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:02:43.066060 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:02:43.066060 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:02:43.066060 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:02:43.066060 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:02:43.066060 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:02:43.066060 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:02:43.066060 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 12:02:43.066060 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 12:02:43.066060 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 25 12:02:43.066060 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jul 25 12:02:43.066060 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 478/952] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Jul 25 12:02:42 2022] GET /identity/v3/services/compute => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:02:43.071613 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbd1cc0-03fc-4b38-8aef-d90db4cef2cf None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:43.121133 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:43.178353 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbd1cc0-03fc-4b38-8aef-d90db4cef2cf None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:43.214319 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0206c62a-a25a-418b-9619-cadbe82e91e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:43.215002 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0206c62a-a25a-418b-9619-cadbe82e91e4 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:43.215296 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0206c62a-a25a-418b-9619-cadbe82e91e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:43.215608 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0206c62a-a25a-418b-9619-cadbe82e91e4 None admin] PATH_INFO: `/v3/services` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:43.216397 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0206c62a-a25a-418b-9619-cadbe82e91e4 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:02:43.216711 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0206c62a-a25a-418b-9619-cadbe82e91e4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:43.218157 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0206c62a-a25a-418b-9619-cadbe82e91e4 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:43.223999 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 475/953] 10.222.0.42 () {56 vars in 1112 bytes} [Mon Jul 25 12:02:43 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:02:43.229346 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2feeab-0859-4e51-a184-5b978220b8f8 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:43.334212 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:43.347531 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2feeab-0859-4e51-a184-5b978220b8f8 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:43.384315 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f879af-aab8-4e08-a94a-a34096f4423d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:43.384994 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f879af-aab8-4e08-a94a-a34096f4423d None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:43.385280 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f879af-aab8-4e08-a94a-a34096f4423d None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:43.385561 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f879af-aab8-4e08-a94a-a34096f4423d None admin] PATH_INFO: `/v3/services` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:43.386268 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f879af-aab8-4e08-a94a-a34096f4423d None admin] RBAC: Adding query filter params (type=compute) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:02:43.386579 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f879af-aab8-4e08-a94a-a34096f4423d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:43.388022 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f879af-aab8-4e08-a94a-a34096f4423d None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:43.393110 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 479/954] 10.222.0.42 () {56 vars in 1112 bytes} [Mon Jul 25 12:02:43 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:02:43.677159 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065b2b17-22b6-469f-87b5-011aa8bf5214 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:43.677929 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065b2b17-22b6-469f-87b5-011aa8bf5214 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:43.678456 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065b2b17-22b6-469f-87b5-011aa8bf5214 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:43.680341 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 476/955] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 25 12:02:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:02:43.685739 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4df186d-fe4e-4e70-a8a7-b4bfa41a49a0 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:43.686315 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4df186d-fe4e-4e70-a8a7-b4bfa41a49a0 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:43.686810 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4df186d-fe4e-4e70-a8a7-b4bfa41a49a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:43.706575 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-a4df186d-fe4e-4e70-a8a7-b4bfa41a49a0 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:02:43.707791 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-a4df186d-fe4e-4e70-a8a7-b4bfa41a49a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:02:43.778456 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 480/956] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 25 12:02:43 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 25 12:02:43.802101 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065b2b17-22b6-469f-87b5-011aa8bf5214 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:43.817120 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6479054-67af-4c6d-b4e1-e75952626366 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:43.817656 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6479054-67af-4c6d-b4e1-e75952626366 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:43.818092 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6479054-67af-4c6d-b4e1-e75952626366 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:43.820536 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 481/957] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 25 12:02:43 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 12:02:43.826653 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a335d4-6098-40f4-bd28-5dfcbc93a324 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:43.827130 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a335d4-6098-40f4-bd28-5dfcbc93a324 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:43.827848 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a335d4-6098-40f4-bd28-5dfcbc93a324 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:43.850288 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-64a335d4-6098-40f4-bd28-5dfcbc93a324 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:02:43.852492 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-64a335d4-6098-40f4-bd28-5dfcbc93a324 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:02:43.917448 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065b2b17-22b6-469f-87b5-011aa8bf5214 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:43.934537 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 482/958] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 25 12:02:43 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 12:02:43.934725 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:43.955288 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a335d4-6098-40f4-bd28-5dfcbc93a324 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:43.969660 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48907ba6-42a4-465e-b57e-e02d5a70a94f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:43.970739 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48907ba6-42a4-465e-b57e-e02d5a70a94f None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:43.971173 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48907ba6-42a4-465e-b57e-e02d5a70a94f None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:43.971628 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48907ba6-42a4-465e-b57e-e02d5a70a94f None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:43.972832 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:43.983741 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48907ba6-42a4-465e-b57e-e02d5a70a94f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 12:02:43.993536 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-48907ba6-42a4-465e-b57e-e02d5a70a94f None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 25 12:02:43.993536 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:02:43.993536 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:02:43.993536 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:02:43.993536 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:02:43.993536 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:02:43.993536 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:02:43.993536 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:02:43.993536 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:02:43.993536 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:02:43.993536 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:02:43.993536 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:02:43.993536 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 12:02:43.993536 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 12:02:43.993536 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 12:02:43.993536 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 12:02:43.994539 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 12:02:43.994539 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 12:02:43.994539 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 12:02:43.994539 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 12:02:43.994539 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:02:43.994539 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:02:43.994539 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:02:43.994539 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:02:43.994539 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:02:43.994539 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:02:43.994539 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:02:43.994539 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 25 12:02:43.994539 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:02:43.994539 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:02:43.994539 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:02:43.994539 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:02:43.995438 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:02:43.995438 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:02:43.995438 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:02:43.995438 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:02:43.995438 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 12:02:43.995438 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 12:02:43.995438 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 12:02:43.995438 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 12:02:43.995438 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 12:02:43.995438 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 12:02:43.995438 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 25 12:02:43.995438 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 25 12:02:43.998207 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 477/959] 10.222.0.42 () {56 vars in 1120 bytes} [Mon Jul 25 12:02:43 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:02:44.005365 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48907ba6-42a4-465e-b57e-e02d5a70a94f None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:44.080158 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a335d4-6098-40f4-bd28-5dfcbc93a324 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:44.119003 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33b3172-509a-46d2-b0ee-d647eacfb5e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:44.120337 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33b3172-509a-46d2-b0ee-d647eacfb5e1 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:44.120841 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33b3172-509a-46d2-b0ee-d647eacfb5e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:44.121334 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33b3172-509a-46d2-b0ee-d647eacfb5e1 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:44.122197 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33b3172-509a-46d2-b0ee-d647eacfb5e1 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:44.122484 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48907ba6-42a4-465e-b57e-e02d5a70a94f None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:44.124204 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33b3172-509a-46d2-b0ee-d647eacfb5e1 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:44.137531 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-c33b3172-509a-46d2-b0ee-d647eacfb5e1 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 25 12:02:44.137531 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:02:44.137531 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 12:02:44.137531 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 12:02:44.137531 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 12:02:44.137531 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 12:02:44.137531 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 12:02:44.137531 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 12:02:44.137531 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 25 12:02:44.137531 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jul 25 12:02:44.137531 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 12:02:44.137531 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jul 25 12:02:44.137531 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:02:44.137531 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:02:44.137531 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:02:44.137531 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:02:44.137531 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:02:44.138691 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:02:44.138691 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:02:44.138691 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:02:44.138691 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:02:44.138691 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:02:44.138691 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:02:44.138691 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 12:02:44.138691 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 12:02:44.138691 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 12:02:44.138691 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 12:02:44.138691 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:02:44.138691 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:02:44.138691 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:02:44.138691 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:02:44.138691 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:02:44.138691 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:02:44.139957 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:02:44.139957 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jul 25 12:02:44.139957 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:02:44.139957 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:02:44.139957 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:02:44.139957 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:02:44.139957 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:02:44.139957 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:02:44.139957 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:02:44.139957 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:02:44.139957 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 12:02:44.139957 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 12:02:44.139957 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 25 12:02:44.139957 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jul 25 12:02:44.142585 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 483/960] 10.222.0.42 () {56 vars in 1126 bytes} [Mon Jul 25 12:02:43 2022] GET /identity/v3/services/object-store => generated 93 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 12:02:44.148832 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33b3172-509a-46d2-b0ee-d647eacfb5e1 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:44.161051 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c3c047-c05a-473a-b856-d441b34d6e15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:44.161413 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c3c047-c05a-473a-b856-d441b34d6e15 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:44.161699 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c3c047-c05a-473a-b856-d441b34d6e15 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:44.161979 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c3c047-c05a-473a-b856-d441b34d6e15 None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:44.162786 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c3c047-c05a-473a-b856-d441b34d6e15 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:02:44.163099 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c3c047-c05a-473a-b856-d441b34d6e15 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:44.164551 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c3c047-c05a-473a-b856-d441b34d6e15 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:44.178655 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 478/961] 10.222.0.42 () {56 vars in 1115 bytes} [Mon Jul 25 12:02:44 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 12:02:44.180369 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c3c047-c05a-473a-b856-d441b34d6e15 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:44.218845 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:44.257470 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:44.289846 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33b3172-509a-46d2-b0ee-d647eacfb5e1 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:44.311944 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c3c047-c05a-473a-b856-d441b34d6e15 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:44.334397 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5587075-f939-43cd-b530-592241f6ac4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:44.335587 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5587075-f939-43cd-b530-592241f6ac4b None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:44.335970 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5587075-f939-43cd-b530-592241f6ac4b None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:44.336345 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5587075-f939-43cd-b530-592241f6ac4b None admin] PATH_INFO: `/v3/services` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:44.337175 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5587075-f939-43cd-b530-592241f6ac4b None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:02:44.337585 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5587075-f939-43cd-b530-592241f6ac4b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:44.339585 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5587075-f939-43cd-b530-592241f6ac4b None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:44.351744 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 484/962] 10.222.0.42 () {56 vars in 1122 bytes} [Mon Jul 25 12:02:44 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 12:02:44.352773 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5587075-f939-43cd-b530-592241f6ac4b None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:44.357140 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4888bd9-130d-49e2-b22c-64052867033d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:44.357938 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4888bd9-130d-49e2-b22c-64052867033d None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:44.358221 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4888bd9-130d-49e2-b22c-64052867033d None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:44.358505 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4888bd9-130d-49e2-b22c-64052867033d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:44.369891 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4888bd9-130d-49e2-b22c-64052867033d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:44.371344 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4888bd9-130d-49e2-b22c-64052867033d None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:44.389999 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-b4888bd9-130d-49e2-b22c-64052867033d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:02:44.389999 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:02:44.389999 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:02:44.389999 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:02:44.389999 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:02:44.389999 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:02:44.389999 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:02:44.389999 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:02:44.389999 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:02:44.389999 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:02:44.389999 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:02:44.389999 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:02:44.389999 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 12:02:44.389999 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 12:02:44.389999 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 12:02:44.391496 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 12:02:44.391496 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:02:44.391496 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:02:44.391496 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:02:44.391496 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:02:44.391496 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:02:44.391496 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:02:44.391496 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:02:44.391496 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 25 12:02:44.391496 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:02:44.391496 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:02:44.391496 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:02:44.391496 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:02:44.391496 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:02:44.391496 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:02:44.391496 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:02:44.391496 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:02:44.392464 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 12:02:44.392464 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 12:02:44.392464 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:02:44.392464 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 25 12:02:44.392464 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 479/963] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jul 25 12:02:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:02:44.393590 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3771e4ef-6cfd-4d6e-a270-f502bec35ec8 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:44.393590 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3771e4ef-6cfd-4d6e-a270-f502bec35ec8 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:44.393590 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3771e4ef-6cfd-4d6e-a270-f502bec35ec8 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:44.398872 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 480/964] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 25 12:02:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:02:44.398872 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3771e4ef-6cfd-4d6e-a270-f502bec35ec8 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:44.471788 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5587075-f939-43cd-b530-592241f6ac4b None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:44.487642 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:44.506632 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcd3eab-feac-4db8-88e8-e8e954b501ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:44.507310 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcd3eab-feac-4db8-88e8-e8e954b501ca None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:44.507629 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcd3eab-feac-4db8-88e8-e8e954b501ca None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:44.507921 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcd3eab-feac-4db8-88e8-e8e954b501ca None admin] PATH_INFO: `/v3/services` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:44.508650 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fcd3eab-feac-4db8-88e8-e8e954b501ca None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:02:44.508972 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fcd3eab-feac-4db8-88e8-e8e954b501ca None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:44.509811 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3771e4ef-6cfd-4d6e-a270-f502bec35ec8 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:44.510389 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fcd3eab-feac-4db8-88e8-e8e954b501ca None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:44.517908 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 485/965] 10.222.0.42 () {56 vars in 1122 bytes} [Mon Jul 25 12:02:44 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:02:44.519102 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c30e26e-2e26-4489-8f38-73e05c408cf7 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:44.519401 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c30e26e-2e26-4489-8f38-73e05c408cf7 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:44.519708 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c30e26e-2e26-4489-8f38-73e05c408cf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:44.527577 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:44.538979 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-6c30e26e-2e26-4489-8f38-73e05c408cf7 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:02:44.540249 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-6c30e26e-2e26-4489-8f38-73e05c408cf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:02:44.558020 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b7e9ce-4462-4941-bada-c98a3e2eab19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:44.559269 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b7e9ce-4462-4941-bada-c98a3e2eab19 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:44.559812 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b7e9ce-4462-4941-bada-c98a3e2eab19 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:44.560297 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b7e9ce-4462-4941-bada-c98a3e2eab19 None admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:44.561250 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b7e9ce-4462-4941-bada-c98a3e2eab19 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:02:44.561777 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b7e9ce-4462-4941-bada-c98a3e2eab19 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:44.564703 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b7e9ce-4462-4941-bada-c98a3e2eab19 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:44.582641 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 481/966] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jul 25 12:02:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 12:02:44.583489 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b7e9ce-4462-4941-bada-c98a3e2eab19 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:44.615253 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 486/967] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 25 12:02:44 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 12:02:44.616390 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c30e26e-2e26-4489-8f38-73e05c408cf7 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:44.692992 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b7e9ce-4462-4941-bada-c98a3e2eab19 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:44.723185 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c30e26e-2e26-4489-8f38-73e05c408cf7 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:44.728347 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d37b43-ac71-45ce-b9b0-9ba8f3371772 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:44.729006 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d37b43-ac71-45ce-b9b0-9ba8f3371772 None admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:44.729293 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d37b43-ac71-45ce-b9b0-9ba8f3371772 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:44.729575 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d37b43-ac71-45ce-b9b0-9ba8f3371772 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:44.730317 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d37b43-ac71-45ce-b9b0-9ba8f3371772 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:44.731713 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d37b43-ac71-45ce-b9b0-9ba8f3371772 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:44.746914 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:44.763579 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 482/968] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Jul 25 12:02:44 2022] POST /identity/v3/endpoints => generated 349 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:02:44.763579 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d37b43-ac71-45ce-b9b0-9ba8f3371772 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:44.771899 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:44.781548 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a8a094-447a-4cd6-8dc9-01a6305b4460 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:44.783942 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a8a094-447a-4cd6-8dc9-01a6305b4460 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:44.784368 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a8a094-447a-4cd6-8dc9-01a6305b4460 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:44.784717 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a8a094-447a-4cd6-8dc9-01a6305b4460 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:44.794151 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a8a094-447a-4cd6-8dc9-01a6305b4460 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:44.796101 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a8a094-447a-4cd6-8dc9-01a6305b4460 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:44.820460 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-36a8a094-447a-4cd6-8dc9-01a6305b4460 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 25 12:02:44.820460 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:02:44.820460 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:02:44.820460 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:02:44.820460 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:02:44.820460 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:02:44.820460 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:02:44.820460 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:02:44.820460 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:02:44.820460 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:02:44.820460 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:02:44.820460 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:02:44.820460 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 12:02:44.820460 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 12:02:44.820460 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 12:02:44.820460 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 12:02:44.821713 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:02:44.821713 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:02:44.821713 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 12:02:44.821713 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 12:02:44.821713 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 12:02:44.821713 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 12:02:44.821713 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:02:44.821713 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:02:44.821713 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:02:44.821713 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:02:44.821713 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:02:44.821713 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jul 25 12:02:44.821713 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:02:44.821713 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:02:44.821713 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:02:44.821713 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:02:44.821713 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:02:44.821713 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:02:44.822887 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:02:44.822887 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:02:44.822887 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 12:02:44.822887 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 12:02:44.822887 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 12:02:44.822887 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 12:02:44.822887 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 12:02:44.822887 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 12:02:44.822887 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 25 12:02:44.822887 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jul 25 12:02:44.827152 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 487/969] 10.222.0.42 () {56 vars in 1117 bytes} [Mon Jul 25 12:02:44 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:02:44.829221 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a8a094-447a-4cd6-8dc9-01a6305b4460 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:44.912596 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d37b43-ac71-45ce-b9b0-9ba8f3371772 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:44.955852 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a8a094-447a-4cd6-8dc9-01a6305b4460 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:44.960043 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383d66c7-6a0c-4f02-aca6-10d912f9c2cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:44.960951 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383d66c7-6a0c-4f02-aca6-10d912f9c2cc None admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:44.961422 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383d66c7-6a0c-4f02-aca6-10d912f9c2cc None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:44.961896 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383d66c7-6a0c-4f02-aca6-10d912f9c2cc None admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:44.962804 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383d66c7-6a0c-4f02-aca6-10d912f9c2cc None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:44.964668 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383d66c7-6a0c-4f02-aca6-10d912f9c2cc None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:44.981807 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.sql.core [None req-383d66c7-6a0c-4f02-aca6-10d912f9c2cc None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 25 12:02:44.981807 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 25 12:02:44.981807 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [parameters: {'user_id': '7bdb51ed69cf4ab582842c16a41153fb', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 25 12:02:44.981807 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80297) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 25 12:02:44.993160 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-383d66c7-6a0c-4f02-aca6-10d912f9c2cc None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 25 12:02:44.993160 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:02:44.993160 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 25 12:02:44.993160 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 25 12:02:44.993160 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 25 12:02:44.993160 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 25 12:02:44.993160 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 25 12:02:44.993160 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application result = self._query(query) Jul 25 12:02:44.993160 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 25 12:02:44.993160 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application conn.query(q) Jul 25 12:02:44.993160 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 25 12:02:44.993160 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 25 12:02:44.993160 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 25 12:02:44.993160 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application result.read() Jul 25 12:02:44.993160 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 25 12:02:44.994105 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 25 12:02:44.994105 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 25 12:02:44.994105 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 25 12:02:44.994105 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 25 12:02:44.994105 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 25 12:02:44.994105 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 25 12:02:44.994105 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 25 12:02:44.994105 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 25 12:02:44.994105 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 25 12:02:44.994105 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 25 12:02:44.994105 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 25 12:02:44.994105 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:02:44.994105 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 25 12:02:44.994105 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 25 12:02:44.994105 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jul 25 12:02:44.994105 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jul 25 12:02:44.994105 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 25 12:02:44.994105 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application next(self.gen) Jul 25 12:02:44.995118 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 25 12:02:44.995118 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application yield resource Jul 25 12:02:44.995118 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 25 12:02:44.995118 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application next(self.gen) Jul 25 12:02:44.995118 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 25 12:02:44.995118 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application self.session.rollback() Jul 25 12:02:44.995118 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 25 12:02:44.995118 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application self.force_reraise() Jul 25 12:02:44.995118 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 25 12:02:44.995118 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise self.value Jul 25 12:02:44.995118 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 25 12:02:44.995118 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 25 12:02:44.995118 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 25 12:02:44.995118 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application session.commit() Jul 25 12:02:44.995118 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 25 12:02:44.995118 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 25 12:02:44.995118 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 25 12:02:44.996262 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application self._prepare_impl() Jul 25 12:02:44.996262 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 25 12:02:44.996262 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application self.session.flush() Jul 25 12:02:44.996262 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 25 12:02:44.996262 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application self._flush(objects) Jul 25 12:02:44.996262 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 25 12:02:44.996262 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 25 12:02:44.996262 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 25 12:02:44.996262 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application compat.raise_( Jul 25 12:02:44.996262 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 25 12:02:44.996262 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception Jul 25 12:02:44.996262 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 25 12:02:44.996262 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application flush_context.execute() Jul 25 12:02:44.996262 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 25 12:02:44.996262 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rec.execute(self) Jul 25 12:02:44.996262 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 25 12:02:44.996262 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 25 12:02:44.997196 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 25 12:02:44.997196 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 25 12:02:44.997196 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jul 25 12:02:44.997196 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application result = connection._execute_20( Jul 25 12:02:44.997196 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 25 12:02:44.997196 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 25 12:02:44.997196 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 25 12:02:44.997196 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 25 12:02:44.997196 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 25 12:02:44.997196 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 25 12:02:44.997196 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 25 12:02:44.997196 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 25 12:02:44.997196 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 25 12:02:44.997196 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 25 12:02:44.997196 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 25 12:02:44.997196 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception Jul 25 12:02:44.998085 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 25 12:02:44.998085 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 25 12:02:44.998085 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 25 12:02:44.998085 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 25 12:02:44.998085 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 25 12:02:44.998085 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application result = self._query(query) Jul 25 12:02:44.998085 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 25 12:02:44.998085 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application conn.query(q) Jul 25 12:02:44.998085 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 25 12:02:44.998085 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 25 12:02:44.998085 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 25 12:02:44.998085 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application result.read() Jul 25 12:02:44.998085 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 25 12:02:44.998085 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 25 12:02:44.998085 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 25 12:02:44.998085 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 25 12:02:44.998928 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 25 12:02:44.998928 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 25 12:02:44.998928 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 25 12:02:44.998928 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 25 12:02:44.998928 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 25 12:02:44.998928 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 25 12:02:44.998928 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [parameters: {'user_id': '7bdb51ed69cf4ab582842c16a41153fb', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 25 12:02:44.998928 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 25 12:02:44.998928 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 25 12:02:44.998928 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 12:02:44.998928 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 25 12:02:44.998928 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:02:44.998928 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:02:44.998928 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:02:44.998928 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:02:45.001680 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:02:45.001680 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:02:45.001680 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:02:45.001680 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:02:45.001680 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:02:45.001680 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:02:45.001680 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:02:45.001680 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jul 25 12:02:45.001680 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jul 25 12:02:45.001680 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:02:45.001680 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:02:45.001680 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 12:02:45.001680 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 12:02:45.001680 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 12:02:45.001680 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 12:02:45.001680 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jul 25 12:02:45.001680 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jul 25 12:02:45.004917 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jul 25 12:02:45.004917 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jul 25 12:02:45.004917 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 25 12:02:45.004917 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 25 12:02:45.004917 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 25 12:02:45.004917 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 25 12:02:45.005764 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff6946f-77c1-4b76-b9b8-50e2b130d095 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:45.005764 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff6946f-77c1-4b76-b9b8-50e2b130d095 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:45.005764 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff6946f-77c1-4b76-b9b8-50e2b130d095 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:45.005764 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff6946f-77c1-4b76-b9b8-50e2b130d095 None admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:45.007879 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff6946f-77c1-4b76-b9b8-50e2b130d095 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:02:45.007879 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff6946f-77c1-4b76-b9b8-50e2b130d095 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:45.007879 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff6946f-77c1-4b76-b9b8-50e2b130d095 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:45.008347 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 483/970] 10.222.0.42 () {60 vars in 1131 bytes} [Mon Jul 25 12:02:44 2022] POST /identity/v3/users => generated 160 bytes in 253 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:02:45.014678 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383d66c7-6a0c-4f02-aca6-10d912f9c2cc None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:45.026869 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 488/971] 10.222.0.42 () {56 vars in 1149 bytes} [Mon Jul 25 12:02:44 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 12:02:45.034683 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff6946f-77c1-4b76-b9b8-50e2b130d095 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:45.101697 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:45.104611 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:45.156547 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383d66c7-6a0c-4f02-aca6-10d912f9c2cc None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:45.179978 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff6946f-77c1-4b76-b9b8-50e2b130d095 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:45.198964 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc0d29c-e9cb-402c-afb6-4f0505460f73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:45.199943 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc0d29c-e9cb-402c-afb6-4f0505460f73 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:45.200236 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc0d29c-e9cb-402c-afb6-4f0505460f73 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:45.200526 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc0d29c-e9cb-402c-afb6-4f0505460f73 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:45.211070 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc0d29c-e9cb-402c-afb6-4f0505460f73 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:45.213277 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc0d29c-e9cb-402c-afb6-4f0505460f73 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:45.225361 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a9a35f-92c6-4746-b28f-03ff5f4c6369 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:45.226360 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a9a35f-92c6-4746-b28f-03ff5f4c6369 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:45.226847 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a9a35f-92c6-4746-b28f-03ff5f4c6369 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:45.227343 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a9a35f-92c6-4746-b28f-03ff5f4c6369 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:45.230274 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-cdc0d29c-e9cb-402c-afb6-4f0505460f73 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 25 12:02:45.230274 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:02:45.230274 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:02:45.230274 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:02:45.230274 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:02:45.230274 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:02:45.230274 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:02:45.230274 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:02:45.230274 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:02:45.230274 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:02:45.230274 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:02:45.230274 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:02:45.230274 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 12:02:45.230274 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 12:02:45.230274 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 12:02:45.230274 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 12:02:45.231228 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:02:45.231228 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:02:45.231228 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 12:02:45.231228 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 12:02:45.231228 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 12:02:45.231228 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 12:02:45.231228 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:02:45.231228 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:02:45.231228 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:02:45.231228 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:02:45.231228 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:02:45.231228 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 25 12:02:45.231228 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:02:45.231228 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:02:45.231228 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:02:45.231228 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:02:45.231228 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:02:45.231228 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:02:45.237187 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:02:45.237187 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:02:45.237187 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 12:02:45.237187 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 12:02:45.237187 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 12:02:45.237187 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 12:02:45.237187 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 12:02:45.237187 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 12:02:45.237187 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 25 12:02:45.237187 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 25 12:02:45.237187 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 484/972] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Jul 25 12:02:45 2022] GET /identity/v3/users/cinder => generated 84 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:02:45.240501 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc0d29c-e9cb-402c-afb6-4f0505460f73 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:45.305856 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a9a35f-92c6-4746-b28f-03ff5f4c6369 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:45.307469 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a9a35f-92c6-4746-b28f-03ff5f4c6369 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:45.349632 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc0d29c-e9cb-402c-afb6-4f0505460f73 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:45.360179 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:45.377765 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:45.394715 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e160c4e-29dd-4fda-920d-e39effda1b57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:45.395806 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e160c4e-29dd-4fda-920d-e39effda1b57 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:45.396397 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e160c4e-29dd-4fda-920d-e39effda1b57 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:45.396893 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e160c4e-29dd-4fda-920d-e39effda1b57 None admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:45.397918 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e160c4e-29dd-4fda-920d-e39effda1b57 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:02:45.398435 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e160c4e-29dd-4fda-920d-e39effda1b57 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:45.400122 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e160c4e-29dd-4fda-920d-e39effda1b57 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:45.419601 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 489/973] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Jul 25 12:02:45 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 25 12:02:45.424114 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a9a35f-92c6-4746-b28f-03ff5f4c6369 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:45.454868 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 485/974] 10.222.0.42 () {56 vars in 1101 bytes} [Mon Jul 25 12:02:45 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:02:45.456204 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf88bd47-2204-4594-b090-6fe9ea05ccc2 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:45.456638 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf88bd47-2204-4594-b090-6fe9ea05ccc2 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:45.456966 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf88bd47-2204-4594-b090-6fe9ea05ccc2 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:45.462378 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 486/975] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 25 12:02:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:02:45.465306 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ba82c9-c26f-410f-91ce-003bc1968447 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:45.465912 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ba82c9-c26f-410f-91ce-003bc1968447 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:45.466403 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ba82c9-c26f-410f-91ce-003bc1968447 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:45.487334 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-b0ba82c9-c26f-410f-91ce-003bc1968447 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:02:45.489470 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-b0ba82c9-c26f-410f-91ce-003bc1968447 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:02:45.541281 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a9a35f-92c6-4746-b28f-03ff5f4c6369 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:45.579290 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 487/976] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 25 12:02:45 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 12:02:45.580010 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664d7dff-eb70-41da-8c86-52f323d5104e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:45.580900 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664d7dff-eb70-41da-8c86-52f323d5104e None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:45.581222 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664d7dff-eb70-41da-8c86-52f323d5104e None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:45.581512 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664d7dff-eb70-41da-8c86-52f323d5104e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:45.593013 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664d7dff-eb70-41da-8c86-52f323d5104e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:45.594547 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664d7dff-eb70-41da-8c86-52f323d5104e None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:45.598337 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ba82c9-c26f-410f-91ce-003bc1968447 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:45.613902 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-664d7dff-eb70-41da-8c86-52f323d5104e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:02:45.613902 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:02:45.613902 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:02:45.613902 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:02:45.613902 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:02:45.613902 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:02:45.613902 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:02:45.613902 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:02:45.613902 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:02:45.613902 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:02:45.613902 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:02:45.613902 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:02:45.613902 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 12:02:45.613902 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 12:02:45.613902 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 12:02:45.615083 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 12:02:45.615083 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:02:45.615083 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:02:45.615083 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:02:45.615083 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:02:45.615083 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:02:45.615083 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:02:45.615083 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:02:45.615083 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jul 25 12:02:45.615083 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:02:45.615083 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:02:45.615083 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:02:45.615083 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:02:45.615083 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:02:45.615083 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:02:45.615083 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:02:45.615083 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:02:45.616288 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 12:02:45.616288 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 12:02:45.616288 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:02:45.616288 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jul 25 12:02:45.616288 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:45.616768 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 490/977] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jul 25 12:02:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 12:02:45.622663 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664d7dff-eb70-41da-8c86-52f323d5104e None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:45.725368 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ba82c9-c26f-410f-91ce-003bc1968447 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:45.740753 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:45.751082 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664d7dff-eb70-41da-8c86-52f323d5104e None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:45.773851 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f811e0a8-d4ba-43c2-adc2-725156fbe1c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:45.774532 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f811e0a8-d4ba-43c2-adc2-725156fbe1c1 None admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:45.777848 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f811e0a8-d4ba-43c2-adc2-725156fbe1c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:45.777848 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f811e0a8-d4ba-43c2-adc2-725156fbe1c1 None admin] PATH_INFO: `/v3/services` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:45.777848 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f811e0a8-d4ba-43c2-adc2-725156fbe1c1 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:45.777848 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f811e0a8-d4ba-43c2-adc2-725156fbe1c1 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:45.794132 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 488/978] 10.222.0.42 () {60 vars in 1141 bytes} [Mon Jul 25 12:02:45 2022] POST /identity/v3/services => generated 242 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 12:02:45.801504 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760d65a7-5244-4527-a8bd-04bbeaa267b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:45.802224 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760d65a7-5244-4527-a8bd-04bbeaa267b3 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:45.802480 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760d65a7-5244-4527-a8bd-04bbeaa267b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:45.802762 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760d65a7-5244-4527-a8bd-04bbeaa267b3 None admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:45.803590 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760d65a7-5244-4527-a8bd-04bbeaa267b3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:02:45.803924 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760d65a7-5244-4527-a8bd-04bbeaa267b3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:45.805635 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760d65a7-5244-4527-a8bd-04bbeaa267b3 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:45.823947 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 491/979] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jul 25 12:02:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:02:45.825674 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f811e0a8-d4ba-43c2-adc2-725156fbe1c1 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:45.939409 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f811e0a8-d4ba-43c2-adc2-725156fbe1c1 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:45.976591 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3142f1-830b-4d88-8339-365f665aa7d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:45.977400 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3142f1-830b-4d88-8339-365f665aa7d7 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:45.977689 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3142f1-830b-4d88-8339-365f665aa7d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:45.977973 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3142f1-830b-4d88-8339-365f665aa7d7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:45.985981 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3142f1-830b-4d88-8339-365f665aa7d7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:45.987434 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3142f1-830b-4d88-8339-365f665aa7d7 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:46.002459 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-dd3142f1-830b-4d88-8339-365f665aa7d7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 12:02:46.002459 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:02:46.002459 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:02:46.002459 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:02:46.002459 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:02:46.002459 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:02:46.002459 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:02:46.002459 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:02:46.002459 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:02:46.002459 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:02:46.002459 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:02:46.002459 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:02:46.002459 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 12:02:46.002459 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 12:02:46.002459 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 12:02:46.003391 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 12:02:46.003391 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:02:46.003391 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:02:46.003391 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:02:46.003391 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:02:46.003391 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:02:46.003391 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:02:46.003391 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:02:46.003391 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 25 12:02:46.003391 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:02:46.003391 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:02:46.003391 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:02:46.003391 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:02:46.003391 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:02:46.003391 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:02:46.003391 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:02:46.003391 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:02:46.004627 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 12:02:46.004627 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 12:02:46.004627 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 12:02:46.004627 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 12:02:46.004627 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 12:02:46.004627 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 12:02:46.004627 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 12:02:46.004627 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 25 12:02:46.004627 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 489/980] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Jul 25 12:02:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 12:02:46.011437 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760d65a7-5244-4527-a8bd-04bbeaa267b3 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:46.070859 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:46.117240 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760d65a7-5244-4527-a8bd-04bbeaa267b3 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:46.151317 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b527667e-a807-46b8-b357-43482cd8953f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:46.152014 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b527667e-a807-46b8-b357-43482cd8953f None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:46.152305 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b527667e-a807-46b8-b357-43482cd8953f None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:46.152589 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b527667e-a807-46b8-b357-43482cd8953f None admin] PATH_INFO: `/v3/projects` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:46.153406 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b527667e-a807-46b8-b357-43482cd8953f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:02:46.153725 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b527667e-a807-46b8-b357-43482cd8953f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:46.155157 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b527667e-a807-46b8-b357-43482cd8953f None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:46.169035 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 492/981] 10.222.0.42 () {56 vars in 1148 bytes} [Mon Jul 25 12:02:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:02:46.175592 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3142f1-830b-4d88-8339-365f665aa7d7 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:46.203637 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:46.276851 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3142f1-830b-4d88-8339-365f665aa7d7 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:46.311585 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d41e8d-d3b2-4dfc-851f-c948c87c1c75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:46.312251 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d41e8d-d3b2-4dfc-851f-c948c87c1c75 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:46.312538 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d41e8d-d3b2-4dfc-851f-c948c87c1c75 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:46.312848 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d41e8d-d3b2-4dfc-851f-c948c87c1c75 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:46.313683 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d41e8d-d3b2-4dfc-851f-c948c87c1c75 None admin] RBAC: Adding query filter params (role.id=3297f946c8f04c9f8a159c7e36bf41a5, scope.project.id=b9f62b8f37304b8a9a4bb54205347577, user.id=93207b858d294d049fa56d45810a40aa) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:02:46.314003 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d41e8d-d3b2-4dfc-851f-c948c87c1c75 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:46.315440 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d41e8d-d3b2-4dfc-851f-c948c87c1c75 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:46.327765 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 490/982] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Jul 25 12:02:46 2022] GET /identity/v3/role_assignments?user.id=93207b858d294d049fa56d45810a40aa&scope.project.id=b9f62b8f37304b8a9a4bb54205347577&role.id=3297f946c8f04c9f8a159c7e36bf41a5 => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:02:46.847427 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282d8ad8-37b4-40e2-b3c0-8649e085ae19 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:46.848001 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282d8ad8-37b4-40e2-b3c0-8649e085ae19 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:46.848001 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282d8ad8-37b4-40e2-b3c0-8649e085ae19 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:46.849554 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 493/983] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 25 12:02:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:02:46.854509 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7027bf8-bc9b-4908-ada5-624838e4c2f4 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:46.854810 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7027bf8-bc9b-4908-ada5-624838e4c2f4 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:46.855098 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7027bf8-bc9b-4908-ada5-624838e4c2f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:46.873133 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-e7027bf8-bc9b-4908-ada5-624838e4c2f4 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:02:46.874327 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-e7027bf8-bc9b-4908-ada5-624838e4c2f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:02:46.945532 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 491/984] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 25 12:02:46 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 12:02:46.968097 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282d8ad8-37b4-40e2-b3c0-8649e085ae19 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:47.072262 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282d8ad8-37b4-40e2-b3c0-8649e085ae19 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:47.089373 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:47.109117 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e2bd58-45d1-474e-af90-639db8a86573 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:47.109973 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e2bd58-45d1-474e-af90-639db8a86573 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:47.110261 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e2bd58-45d1-474e-af90-639db8a86573 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:47.110545 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e2bd58-45d1-474e-af90-639db8a86573 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:47.111204 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e2bd58-45d1-474e-af90-639db8a86573 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:47.112607 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e2bd58-45d1-474e-af90-639db8a86573 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:47.124366 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-16e2bd58-45d1-474e-af90-639db8a86573 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 25 12:02:47.124366 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:02:47.124366 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 12:02:47.124366 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 12:02:47.124366 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 12:02:47.124366 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 12:02:47.124366 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 12:02:47.124366 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 12:02:47.124366 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 25 12:02:47.124366 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jul 25 12:02:47.124366 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 12:02:47.124366 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jul 25 12:02:47.124366 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:02:47.124366 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:02:47.124366 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:02:47.124366 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:02:47.124366 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:02:47.125868 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:02:47.125868 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:02:47.125868 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:02:47.125868 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:02:47.125868 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:02:47.125868 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:02:47.125868 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 12:02:47.125868 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 12:02:47.125868 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 12:02:47.125868 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 12:02:47.125868 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:02:47.125868 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:02:47.125868 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:02:47.125868 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:02:47.125868 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:02:47.125868 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:02:47.126708 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:02:47.126708 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jul 25 12:02:47.126708 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:02:47.126708 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:02:47.126708 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:02:47.126708 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:02:47.126708 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:02:47.126708 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:02:47.126708 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:02:47.126708 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:02:47.126708 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 12:02:47.126708 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 12:02:47.126708 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 25 12:02:47.126708 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jul 25 12:02:47.128116 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 494/985] 10.222.0.42 () {56 vars in 1126 bytes} [Mon Jul 25 12:02:46 2022] GET /identity/v3/services/object-store => generated 93 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:02:47.132064 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7027bf8-bc9b-4908-ada5-624838e4c2f4 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:47.156513 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:47.231500 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7027bf8-bc9b-4908-ada5-624838e4c2f4 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:47.264982 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92be69cd-5b22-4dfb-bab8-da48962495f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:47.265647 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92be69cd-5b22-4dfb-bab8-da48962495f9 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:47.266502 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92be69cd-5b22-4dfb-bab8-da48962495f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:47.266787 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92be69cd-5b22-4dfb-bab8-da48962495f9 None admin] PATH_INFO: `/v3/services` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:47.267497 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92be69cd-5b22-4dfb-bab8-da48962495f9 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:02:47.267849 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92be69cd-5b22-4dfb-bab8-da48962495f9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:47.269225 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92be69cd-5b22-4dfb-bab8-da48962495f9 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:47.278712 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 492/986] 10.222.0.42 () {56 vars in 1122 bytes} [Mon Jul 25 12:02:47 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:02:47.280509 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e2bd58-45d1-474e-af90-639db8a86573 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:47.384551 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e2bd58-45d1-474e-af90-639db8a86573 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:47.422672 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcfda34-b392-4fe4-9e1d-ac105c399bb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:47.423345 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcfda34-b392-4fe4-9e1d-ac105c399bb4 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:47.423669 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcfda34-b392-4fe4-9e1d-ac105c399bb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:47.423956 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcfda34-b392-4fe4-9e1d-ac105c399bb4 None admin] PATH_INFO: `/v3/services` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:47.424665 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcfda34-b392-4fe4-9e1d-ac105c399bb4 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:02:47.424980 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcfda34-b392-4fe4-9e1d-ac105c399bb4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:47.426343 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcfda34-b392-4fe4-9e1d-ac105c399bb4 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:47.436243 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 495/987] 10.222.0.42 () {56 vars in 1122 bytes} [Mon Jul 25 12:02:47 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:02:47.438401 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92be69cd-5b22-4dfb-bab8-da48962495f9 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:47.529256 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:47.545922 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92be69cd-5b22-4dfb-bab8-da48962495f9 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:47.581426 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec4b4ed-3324-4638-8b0c-922799c3a1ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:47.582092 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec4b4ed-3324-4638-8b0c-922799c3a1ce None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:47.582385 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec4b4ed-3324-4638-8b0c-922799c3a1ce None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:47.582673 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec4b4ed-3324-4638-8b0c-922799c3a1ce None admin] PATH_INFO: `/v3/endpoints` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:47.583477 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec4b4ed-3324-4638-8b0c-922799c3a1ce None admin] RBAC: Adding query filter params (interface=admin, service_id=5b1edfdedd42440e93e3c22748838045, region_id=RegionOne) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:02:47.583829 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec4b4ed-3324-4638-8b0c-922799c3a1ce None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:47.585204 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec4b4ed-3324-4638-8b0c-922799c3a1ce None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:47.594749 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 493/988] 10.222.0.42 () {56 vars in 1249 bytes} [Mon Jul 25 12:02:47 2022] GET /identity/v3/endpoints?service_id=5b1edfdedd42440e93e3c22748838045&interface=admin®ion_id=RegionOne => generated 193 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 12:02:47.596386 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcfda34-b392-4fe4-9e1d-ac105c399bb4 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:47.644296 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:47.702184 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcfda34-b392-4fe4-9e1d-ac105c399bb4 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:47.736640 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e16159-577d-41b3-8e03-71250665fbbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:47.737303 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e16159-577d-41b3-8e03-71250665fbbb None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:47.737591 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e16159-577d-41b3-8e03-71250665fbbb None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:47.737874 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e16159-577d-41b3-8e03-71250665fbbb None admin] PATH_INFO: `/v3/services` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:47.738555 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e16159-577d-41b3-8e03-71250665fbbb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:47.739955 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e16159-577d-41b3-8e03-71250665fbbb None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:47.751790 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 496/989] 10.222.0.42 () {56 vars in 1087 bytes} [Mon Jul 25 12:02:47 2022] GET /identity/v3/services => generated 1727 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 12:02:47.838914 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724f78de-e139-4ff9-a0a9-b35df8777016 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:47.839266 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724f78de-e139-4ff9-a0a9-b35df8777016 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:47.839484 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724f78de-e139-4ff9-a0a9-b35df8777016 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:47.841119 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 494/990] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 25 12:02:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:02:47.847035 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab721efc-1a9c-4b4b-b3f5-eafb21f7f562 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:47.847332 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab721efc-1a9c-4b4b-b3f5-eafb21f7f562 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:47.847656 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab721efc-1a9c-4b4b-b3f5-eafb21f7f562 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:47.867180 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-ab721efc-1a9c-4b4b-b3f5-eafb21f7f562 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:02:47.868433 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-ab721efc-1a9c-4b4b-b3f5-eafb21f7f562 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:02:47.940443 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 497/991] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 25 12:02:47 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 12:02:47.962345 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724f78de-e139-4ff9-a0a9-b35df8777016 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:48.070248 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724f78de-e139-4ff9-a0a9-b35df8777016 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:48.104774 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0e1998-7a13-4d6c-9d94-11e10f2c4aad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:48.105576 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0e1998-7a13-4d6c-9d94-11e10f2c4aad None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:48.105863 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0e1998-7a13-4d6c-9d94-11e10f2c4aad None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:48.106143 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0e1998-7a13-4d6c-9d94-11e10f2c4aad None admin] PATH_INFO: `/v3/services/compute` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:48.106796 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0e1998-7a13-4d6c-9d94-11e10f2c4aad None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:48.108202 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0e1998-7a13-4d6c-9d94-11e10f2c4aad None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:48.120427 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-4c0e1998-7a13-4d6c-9d94-11e10f2c4aad None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 25 12:02:48.120427 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:02:48.120427 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 12:02:48.120427 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 12:02:48.120427 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 12:02:48.120427 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 12:02:48.120427 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 12:02:48.120427 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 12:02:48.120427 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 25 12:02:48.120427 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 25 12:02:48.120427 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 12:02:48.120427 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 25 12:02:48.120427 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:02:48.120427 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:02:48.120427 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:02:48.120427 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:02:48.120427 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:02:48.121465 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:02:48.121465 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:02:48.121465 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:02:48.121465 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:02:48.121465 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:02:48.121465 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:02:48.121465 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 12:02:48.121465 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 12:02:48.121465 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 12:02:48.121465 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 12:02:48.121465 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:02:48.121465 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:02:48.121465 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:02:48.121465 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:02:48.121465 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:02:48.121465 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:02:48.122340 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:02:48.122340 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 25 12:02:48.122340 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:02:48.122340 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:02:48.122340 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:02:48.122340 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:02:48.122340 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:02:48.122340 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:02:48.122340 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:02:48.122340 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:02:48.122340 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 12:02:48.122340 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 12:02:48.122340 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 25 12:02:48.122340 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 25 12:02:48.126545 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 495/992] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Jul 25 12:02:47 2022] GET /identity/v3/services/compute => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:02:48.127919 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab721efc-1a9c-4b4b-b3f5-eafb21f7f562 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:48.171655 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:48.233660 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab721efc-1a9c-4b4b-b3f5-eafb21f7f562 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:48.267287 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89af324b-5eb0-44c8-9356-0b1f5a5d10c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:48.267986 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89af324b-5eb0-44c8-9356-0b1f5a5d10c1 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:48.268273 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89af324b-5eb0-44c8-9356-0b1f5a5d10c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:48.268553 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89af324b-5eb0-44c8-9356-0b1f5a5d10c1 None admin] PATH_INFO: `/v3/services` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:48.269260 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89af324b-5eb0-44c8-9356-0b1f5a5d10c1 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:02:48.269572 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89af324b-5eb0-44c8-9356-0b1f5a5d10c1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:48.270940 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89af324b-5eb0-44c8-9356-0b1f5a5d10c1 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:48.279783 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 498/993] 10.222.0.42 () {56 vars in 1112 bytes} [Mon Jul 25 12:02:48 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:02:48.282798 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0e1998-7a13-4d6c-9d94-11e10f2c4aad None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:48.300081 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:48.357262 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e51c8ea-2595-4b44-8a39-743ba6b81d0d None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:48.358324 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e51c8ea-2595-4b44-8a39-743ba6b81d0d None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:48.358629 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e51c8ea-2595-4b44-8a39-743ba6b81d0d None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:48.360204 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 499/994] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 25 12:02:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 12:02:48.365068 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa2620d-6b3d-4d76-b32f-451e0eab0d12 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:48.365367 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa2620d-6b3d-4d76-b32f-451e0eab0d12 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:48.365655 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa2620d-6b3d-4d76-b32f-451e0eab0d12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:48.384334 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-9aa2620d-6b3d-4d76-b32f-451e0eab0d12 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:02:48.385520 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-9aa2620d-6b3d-4d76-b32f-451e0eab0d12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:02:48.392801 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0e1998-7a13-4d6c-9d94-11e10f2c4aad None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:48.429032 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538ec82b-406f-43bb-8540-d06d420ff01f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:48.429821 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538ec82b-406f-43bb-8540-d06d420ff01f None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:48.430120 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538ec82b-406f-43bb-8540-d06d420ff01f None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:48.430407 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538ec82b-406f-43bb-8540-d06d420ff01f None admin] PATH_INFO: `/v3/services` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:48.431124 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538ec82b-406f-43bb-8540-d06d420ff01f None admin] RBAC: Adding query filter params (type=compute) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:02:48.431448 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538ec82b-406f-43bb-8540-d06d420ff01f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:48.432880 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538ec82b-406f-43bb-8540-d06d420ff01f None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:48.438943 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 496/995] 10.222.0.42 () {56 vars in 1112 bytes} [Mon Jul 25 12:02:48 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:02:48.444384 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538ec82b-406f-43bb-8540-d06d420ff01f None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:48.465099 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 500/996] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 25 12:02:48 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 25 12:02:48.486318 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa2620d-6b3d-4d76-b32f-451e0eab0d12 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:48.519500 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:48.529750 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:48.553359 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538ec82b-406f-43bb-8540-d06d420ff01f None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:48.589035 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f7f138-5701-4f8d-a59d-d9e668fd4ad3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:48.589703 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f7f138-5701-4f8d-a59d-d9e668fd4ad3 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:48.589995 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f7f138-5701-4f8d-a59d-d9e668fd4ad3 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:48.590282 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f7f138-5701-4f8d-a59d-d9e668fd4ad3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:48.591079 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f7f138-5701-4f8d-a59d-d9e668fd4ad3 None admin] RBAC: Adding query filter params (interface=public, service_id=f2ce8a88e0aa499ea820b0833e6f2f60, region_id=RegionOne) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:02:48.591408 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f7f138-5701-4f8d-a59d-d9e668fd4ad3 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:48.593685 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f7f138-5701-4f8d-a59d-d9e668fd4ad3 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:48.599069 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 497/997] 10.222.0.42 () {56 vars in 1251 bytes} [Mon Jul 25 12:02:48 2022] GET /identity/v3/endpoints?service_id=f2ce8a88e0aa499ea820b0833e6f2f60&interface=public®ion_id=RegionOne => generated 194 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 12:02:48.599384 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa2620d-6b3d-4d76-b32f-451e0eab0d12 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:48.604359 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f7f138-5701-4f8d-a59d-d9e668fd4ad3 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:48.638682 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f607da8b-036b-4dea-bacb-30d2c8da3aff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:48.639476 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f607da8b-036b-4dea-bacb-30d2c8da3aff None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:48.639795 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f607da8b-036b-4dea-bacb-30d2c8da3aff None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:48.640092 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f607da8b-036b-4dea-bacb-30d2c8da3aff None admin] PATH_INFO: `/v3/services/image` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:48.641339 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f607da8b-036b-4dea-bacb-30d2c8da3aff None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:48.642709 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f607da8b-036b-4dea-bacb-30d2c8da3aff None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:48.654075 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-f607da8b-036b-4dea-bacb-30d2c8da3aff None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 25 12:02:48.654075 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:02:48.654075 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 12:02:48.654075 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 12:02:48.654075 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 12:02:48.654075 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 12:02:48.654075 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 12:02:48.654075 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 12:02:48.654075 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 25 12:02:48.654075 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jul 25 12:02:48.654075 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 12:02:48.654075 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jul 25 12:02:48.654075 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:02:48.654075 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:02:48.654075 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:02:48.654075 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:02:48.654075 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:02:48.655057 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:02:48.655057 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:02:48.655057 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:02:48.655057 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:02:48.655057 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:02:48.655057 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:02:48.655057 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 12:02:48.655057 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 12:02:48.655057 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 12:02:48.655057 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 12:02:48.655057 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:02:48.655057 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:02:48.655057 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:02:48.655057 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:02:48.655057 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:02:48.655057 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:02:48.656170 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:02:48.656170 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jul 25 12:02:48.656170 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:02:48.656170 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:02:48.656170 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:02:48.656170 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:02:48.656170 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:02:48.656170 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:02:48.656170 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:02:48.656170 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:02:48.656170 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 12:02:48.656170 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 12:02:48.656170 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 25 12:02:48.656170 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jul 25 12:02:48.656170 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 501/998] 10.222.0.42 () {56 vars in 1105 bytes} [Mon Jul 25 12:02:48 2022] GET /identity/v3/services/image => generated 86 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 12:02:48.661074 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f607da8b-036b-4dea-bacb-30d2c8da3aff None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:48.714590 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f7f138-5701-4f8d-a59d-d9e668fd4ad3 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:48.753418 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00772352-50f9-4e21-9af0-50d170d9f46d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:48.754165 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00772352-50f9-4e21-9af0-50d170d9f46d None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:48.754458 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00772352-50f9-4e21-9af0-50d170d9f46d None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:48.755049 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00772352-50f9-4e21-9af0-50d170d9f46d None admin] PATH_INFO: `/v3/services` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:48.756028 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00772352-50f9-4e21-9af0-50d170d9f46d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:48.757704 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:48.757883 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00772352-50f9-4e21-9af0-50d170d9f46d None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:48.765241 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 498/999] 10.222.0.42 () {56 vars in 1087 bytes} [Mon Jul 25 12:02:48 2022] GET /identity/v3/services => generated 1727 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:02:48.765634 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f607da8b-036b-4dea-bacb-30d2c8da3aff None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:48.811059 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd69cdc-2c25-4380-a09a-12b63ff19abc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:48.811790 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd69cdc-2c25-4380-a09a-12b63ff19abc None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:48.812086 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd69cdc-2c25-4380-a09a-12b63ff19abc None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:48.812372 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd69cdc-2c25-4380-a09a-12b63ff19abc None admin] PATH_INFO: `/v3/services` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:48.813757 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd69cdc-2c25-4380-a09a-12b63ff19abc None admin] RBAC: Adding query filter params (name=image) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:02:48.814084 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd69cdc-2c25-4380-a09a-12b63ff19abc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:48.815520 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd69cdc-2c25-4380-a09a-12b63ff19abc None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:48.821631 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 502/1000] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jul 25 12:02:48 2022] GET /identity/v3/services?name=image => generated 122 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 12:02:48.827241 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00772352-50f9-4e21-9af0-50d170d9f46d None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:48.838522 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:48.943698 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00772352-50f9-4e21-9af0-50d170d9f46d None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:48.977530 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b734d26-8aa7-429f-b656-4e1b0d392f92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:48.978198 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b734d26-8aa7-429f-b656-4e1b0d392f92 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:48.978490 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b734d26-8aa7-429f-b656-4e1b0d392f92 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:48.978777 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b734d26-8aa7-429f-b656-4e1b0d392f92 None admin] PATH_INFO: `/v3/services` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:48.979491 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b734d26-8aa7-429f-b656-4e1b0d392f92 None admin] RBAC: Adding query filter params (type=image) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:02:48.979828 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b734d26-8aa7-429f-b656-4e1b0d392f92 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:48.981210 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b734d26-8aa7-429f-b656-4e1b0d392f92 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:48.986305 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 499/1001] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jul 25 12:02:48 2022] GET /identity/v3/services?type=image => generated 122 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 12:02:49.750189 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef4e01b-ab8d-48f5-bf63-4d73434d1140 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:49.750708 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef4e01b-ab8d-48f5-bf63-4d73434d1140 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:49.750786 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef4e01b-ab8d-48f5-bf63-4d73434d1140 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:49.752313 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 503/1002] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 25 12:02:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:02:49.757146 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e259a9b6-a14c-475b-9f5f-1bedcc9d045d None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:49.757445 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e259a9b6-a14c-475b-9f5f-1bedcc9d045d None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:49.757730 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e259a9b6-a14c-475b-9f5f-1bedcc9d045d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:49.776953 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-e259a9b6-a14c-475b-9f5f-1bedcc9d045d None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:02:49.778547 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-e259a9b6-a14c-475b-9f5f-1bedcc9d045d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:02:49.843295 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:49.851348 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 500/1003] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 25 12:02:49 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 12:02:49.870860 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef4e01b-ab8d-48f5-bf63-4d73434d1140 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:49.976006 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef4e01b-ab8d-48f5-bf63-4d73434d1140 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:50.008963 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658f09c6-bdc4-4b2a-b634-462e7c47a0eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:50.009696 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658f09c6-bdc4-4b2a-b634-462e7c47a0eb None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:50.009981 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658f09c6-bdc4-4b2a-b634-462e7c47a0eb None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:50.010263 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658f09c6-bdc4-4b2a-b634-462e7c47a0eb None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:50.010918 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658f09c6-bdc4-4b2a-b634-462e7c47a0eb None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:50.012317 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658f09c6-bdc4-4b2a-b634-462e7c47a0eb None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:50.024850 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-658f09c6-bdc4-4b2a-b634-462e7c47a0eb None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 25 12:02:50.024850 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:02:50.024850 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 12:02:50.024850 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 12:02:50.024850 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 12:02:50.024850 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 12:02:50.024850 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 12:02:50.024850 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 12:02:50.024850 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 25 12:02:50.024850 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jul 25 12:02:50.024850 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 12:02:50.024850 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jul 25 12:02:50.024850 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:02:50.024850 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:02:50.024850 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:02:50.024850 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:02:50.024850 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:02:50.025835 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:02:50.025835 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:02:50.025835 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:02:50.025835 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:02:50.025835 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:02:50.025835 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:02:50.025835 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 12:02:50.025835 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 12:02:50.025835 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 12:02:50.025835 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 12:02:50.025835 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:02:50.025835 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:02:50.025835 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:02:50.025835 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:02:50.025835 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:02:50.025835 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:02:50.026673 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:02:50.026673 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jul 25 12:02:50.026673 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:02:50.026673 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:02:50.026673 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:02:50.026673 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:02:50.026673 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:02:50.026673 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:02:50.026673 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:02:50.026673 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:02:50.026673 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 12:02:50.026673 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 12:02:50.026673 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 25 12:02:50.026673 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jul 25 12:02:50.028076 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 504/1004] 10.222.0.42 () {56 vars in 1126 bytes} [Mon Jul 25 12:02:49 2022] GET /identity/v3/services/object-store => generated 93 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:02:50.032033 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e259a9b6-a14c-475b-9f5f-1bedcc9d045d None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:50.133299 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e259a9b6-a14c-475b-9f5f-1bedcc9d045d None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:50.167446 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e398da49-9236-46bd-bbf7-e75ff7ba6d90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:50.168182 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e398da49-9236-46bd-bbf7-e75ff7ba6d90 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:50.168471 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e398da49-9236-46bd-bbf7-e75ff7ba6d90 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:50.168752 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e398da49-9236-46bd-bbf7-e75ff7ba6d90 None admin] PATH_INFO: `/v3/services` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:50.169465 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e398da49-9236-46bd-bbf7-e75ff7ba6d90 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:02:50.169777 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e398da49-9236-46bd-bbf7-e75ff7ba6d90 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:50.171148 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e398da49-9236-46bd-bbf7-e75ff7ba6d90 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:50.176439 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 501/1005] 10.222.0.42 () {56 vars in 1122 bytes} [Mon Jul 25 12:02:50 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 12:02:50.181929 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658f09c6-bdc4-4b2a-b634-462e7c47a0eb None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:50.209907 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:50.286690 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658f09c6-bdc4-4b2a-b634-462e7c47a0eb None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:50.320926 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b390d38c-41ce-4603-87cd-37491c0860f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:50.321623 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b390d38c-41ce-4603-87cd-37491c0860f3 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:50.321917 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b390d38c-41ce-4603-87cd-37491c0860f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:50.322204 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b390d38c-41ce-4603-87cd-37491c0860f3 None admin] PATH_INFO: `/v3/services` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:50.322911 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b390d38c-41ce-4603-87cd-37491c0860f3 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:02:50.323225 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b390d38c-41ce-4603-87cd-37491c0860f3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:50.324620 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b390d38c-41ce-4603-87cd-37491c0860f3 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:50.334373 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 505/1006] 10.222.0.42 () {56 vars in 1122 bytes} [Mon Jul 25 12:02:50 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:02:50.335438 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e398da49-9236-46bd-bbf7-e75ff7ba6d90 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:50.399614 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:50.440262 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e398da49-9236-46bd-bbf7-e75ff7ba6d90 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:50.475499 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e928a9-6cf6-4fe9-adeb-d4682020715e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:50.476558 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e928a9-6cf6-4fe9-adeb-d4682020715e None admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:50.477081 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e928a9-6cf6-4fe9-adeb-d4682020715e None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:50.477555 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e928a9-6cf6-4fe9-adeb-d4682020715e None admin] PATH_INFO: `/v3/endpoints` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:50.478486 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e928a9-6cf6-4fe9-adeb-d4682020715e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:50.480067 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e928a9-6cf6-4fe9-adeb-d4682020715e None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:50.499570 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 502/1007] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Jul 25 12:02:50 2022] POST /identity/v3/endpoints => generated 325 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:02:50.781845 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3730c3b3-7df0-4674-9748-40a30763c0a7 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:50.782235 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3730c3b3-7df0-4674-9748-40a30763c0a7 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:50.782432 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3730c3b3-7df0-4674-9748-40a30763c0a7 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:50.787440 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 506/1008] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 25 12:02:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:02:50.789090 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f41b40-976e-4248-b24c-162414f86863 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:50.789386 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f41b40-976e-4248-b24c-162414f86863 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:50.789669 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f41b40-976e-4248-b24c-162414f86863 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:50.808125 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-56f41b40-976e-4248-b24c-162414f86863 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:02:50.809318 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-56f41b40-976e-4248-b24c-162414f86863 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:02:50.883148 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 503/1009] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 25 12:02:50 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 12:02:50.905664 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3730c3b3-7df0-4674-9748-40a30763c0a7 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:50.995602 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ac39d0-57af-419d-9e7a-7f4f653b8b57 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:50.995899 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ac39d0-57af-419d-9e7a-7f4f653b8b57 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:50.996174 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ac39d0-57af-419d-9e7a-7f4f653b8b57 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:50.996282 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:50.997610 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 504/1010] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 25 12:02:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 12:02:51.003054 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aced2f78-7956-428a-8aaf-1078bdb3ed83 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:51.003419 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aced2f78-7956-428a-8aaf-1078bdb3ed83 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:51.003874 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aced2f78-7956-428a-8aaf-1078bdb3ed83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:51.011022 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3730c3b3-7df0-4674-9748-40a30763c0a7 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:51.022814 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-aced2f78-7956-428a-8aaf-1078bdb3ed83 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:02:51.024045 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-aced2f78-7956-428a-8aaf-1078bdb3ed83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:02:51.051688 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701bbff4-e2f0-49ca-85f8-368a10a4aa31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:51.052523 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701bbff4-e2f0-49ca-85f8-368a10a4aa31 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:51.052812 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701bbff4-e2f0-49ca-85f8-368a10a4aa31 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:51.053094 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701bbff4-e2f0-49ca-85f8-368a10a4aa31 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:51.053868 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701bbff4-e2f0-49ca-85f8-368a10a4aa31 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:51.055464 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701bbff4-e2f0-49ca-85f8-368a10a4aa31 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:51.068604 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-701bbff4-e2f0-49ca-85f8-368a10a4aa31 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 25 12:02:51.068604 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:02:51.068604 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 12:02:51.068604 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 12:02:51.068604 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 12:02:51.068604 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 12:02:51.068604 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 12:02:51.068604 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 12:02:51.068604 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 25 12:02:51.068604 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jul 25 12:02:51.068604 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 12:02:51.068604 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jul 25 12:02:51.068604 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:02:51.068604 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:02:51.068604 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:02:51.068604 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:02:51.068604 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:02:51.069608 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:02:51.069608 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:02:51.069608 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:02:51.069608 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:02:51.069608 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:02:51.069608 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:02:51.069608 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 12:02:51.069608 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 12:02:51.069608 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 12:02:51.069608 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 12:02:51.069608 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:02:51.069608 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:02:51.069608 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:02:51.069608 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:02:51.069608 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:02:51.069608 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:02:51.070477 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:02:51.070477 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jul 25 12:02:51.070477 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:02:51.070477 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:02:51.070477 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:02:51.070477 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:02:51.070477 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:02:51.070477 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:02:51.070477 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:02:51.070477 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:02:51.070477 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 12:02:51.070477 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 12:02:51.070477 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 25 12:02:51.070477 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jul 25 12:02:51.070477 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 507/1011] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Jul 25 12:02:50 2022] GET /identity/v3/services/compute => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:02:51.073947 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:51.075689 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701bbff4-e2f0-49ca-85f8-368a10a4aa31 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:51.099669 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 505/1012] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 25 12:02:51 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 25 12:02:51.122271 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aced2f78-7956-428a-8aaf-1078bdb3ed83 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:51.198835 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701bbff4-e2f0-49ca-85f8-368a10a4aa31 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:51.228392 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aced2f78-7956-428a-8aaf-1078bdb3ed83 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:51.235645 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fe8653-40ca-4812-a19b-dde49ab66bfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:51.236435 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fe8653-40ca-4812-a19b-dde49ab66bfb None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:51.236723 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fe8653-40ca-4812-a19b-dde49ab66bfb None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:51.237005 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fe8653-40ca-4812-a19b-dde49ab66bfb None admin] PATH_INFO: `/v3/services` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:51.237713 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fe8653-40ca-4812-a19b-dde49ab66bfb None admin] RBAC: Adding query filter params (name=compute) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:02:51.238149 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fe8653-40ca-4812-a19b-dde49ab66bfb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:51.239650 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fe8653-40ca-4812-a19b-dde49ab66bfb None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:51.244852 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 508/1013] 10.222.0.42 () {56 vars in 1112 bytes} [Mon Jul 25 12:02:51 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:02:51.250913 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fe8653-40ca-4812-a19b-dde49ab66bfb None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:51.269548 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:51.269717 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343c2082-f296-475a-81b3-0d9451822abf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:51.270215 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343c2082-f296-475a-81b3-0d9451822abf None admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:51.270503 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343c2082-f296-475a-81b3-0d9451822abf None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:51.270779 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343c2082-f296-475a-81b3-0d9451822abf None admin] PATH_INFO: `/v3/services` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:51.271505 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343c2082-f296-475a-81b3-0d9451822abf None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:51.272905 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343c2082-f296-475a-81b3-0d9451822abf None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:51.290261 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 506/1014] 10.222.0.42 () {60 vars in 1141 bytes} [Mon Jul 25 12:02:51 2022] POST /identity/v3/services => generated 242 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 12:02:51.372398 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fe8653-40ca-4812-a19b-dde49ab66bfb None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:51.420691 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47ba688-9745-4c43-88cd-c980b2edb4b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:51.421389 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47ba688-9745-4c43-88cd-c980b2edb4b1 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:51.421678 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47ba688-9745-4c43-88cd-c980b2edb4b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:51.421992 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47ba688-9745-4c43-88cd-c980b2edb4b1 None admin] PATH_INFO: `/v3/services` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:51.422715 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47ba688-9745-4c43-88cd-c980b2edb4b1 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:02:51.423031 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47ba688-9745-4c43-88cd-c980b2edb4b1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:51.424483 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47ba688-9745-4c43-88cd-c980b2edb4b1 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:51.430311 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 509/1015] 10.222.0.42 () {56 vars in 1112 bytes} [Mon Jul 25 12:02:51 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 12:02:51.435843 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343c2082-f296-475a-81b3-0d9451822abf None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:51.484409 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:51.549636 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343c2082-f296-475a-81b3-0d9451822abf None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:51.584100 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b4f44c-7b17-424b-b55e-2b11e6a75258 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:51.584770 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b4f44c-7b17-424b-b55e-2b11e6a75258 None admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:51.585067 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b4f44c-7b17-424b-b55e-2b11e6a75258 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:51.585351 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b4f44c-7b17-424b-b55e-2b11e6a75258 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:51.586111 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b4f44c-7b17-424b-b55e-2b11e6a75258 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:51.587666 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b4f44c-7b17-424b-b55e-2b11e6a75258 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:51.607354 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 507/1016] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Jul 25 12:02:51 2022] POST /identity/v3/endpoints => generated 334 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 12:02:52.531931 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba13756-ecad-4fbb-8de8-f923d7d125ff None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:52.532501 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba13756-ecad-4fbb-8de8-f923d7d125ff None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:52.532501 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba13756-ecad-4fbb-8de8-f923d7d125ff None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:52.533850 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 510/1017] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 25 12:02:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:02:52.539221 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1564ae-aeda-4bce-8506-1a3d566c7721 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:52.539676 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1564ae-aeda-4bce-8506-1a3d566c7721 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:52.540125 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1564ae-aeda-4bce-8506-1a3d566c7721 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:52.559281 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-dd1564ae-aeda-4bce-8506-1a3d566c7721 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:02:52.560470 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-dd1564ae-aeda-4bce-8506-1a3d566c7721 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:02:52.635582 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 508/1018] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 25 12:02:52 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 12:02:52.657773 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba13756-ecad-4fbb-8de8-f923d7d125ff None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:52.736521 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:52.761333 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba13756-ecad-4fbb-8de8-f923d7d125ff None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:52.804694 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fe23ff-82d7-4c59-b755-d8ce7812dfc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:52.805523 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fe23ff-82d7-4c59-b755-d8ce7812dfc2 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:52.805815 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fe23ff-82d7-4c59-b755-d8ce7812dfc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:52.806101 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fe23ff-82d7-4c59-b755-d8ce7812dfc2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:52.807443 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fe23ff-82d7-4c59-b755-d8ce7812dfc2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:52.808953 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fe23ff-82d7-4c59-b755-d8ce7812dfc2 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:52.810871 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 511/1019] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jul 25 12:02:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:02:52.816886 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1564ae-aeda-4bce-8506-1a3d566c7721 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:52.841732 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:52.921030 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1564ae-aeda-4bce-8506-1a3d566c7721 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:52.955228 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e97bf1-cf74-478b-b5d0-f367b1259e4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:52.955924 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e97bf1-cf74-478b-b5d0-f367b1259e4c None admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:52.956215 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e97bf1-cf74-478b-b5d0-f367b1259e4c None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:52.956500 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e97bf1-cf74-478b-b5d0-f367b1259e4c None admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:52.957224 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e97bf1-cf74-478b-b5d0-f367b1259e4c None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:52.958667 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e97bf1-cf74-478b-b5d0-f367b1259e4c None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:52.986823 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 509/1020] 10.222.0.42 () {60 vars in 1141 bytes} [Mon Jul 25 12:02:52 2022] POST /identity/v3/projects => generated 311 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:02:53.365588 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57fab62-ab11-413a-a821-3f2c67b0daeb None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:53.365993 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57fab62-ab11-413a-a821-3f2c67b0daeb None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:53.366267 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57fab62-ab11-413a-a821-3f2c67b0daeb None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:53.367930 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 512/1021] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 25 12:02:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:02:53.373397 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c9b89b-28a9-48f7-98c0-be7b2fbd7efb None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:53.373701 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c9b89b-28a9-48f7-98c0-be7b2fbd7efb None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:53.373995 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c9b89b-28a9-48f7-98c0-be7b2fbd7efb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:53.393161 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-65c9b89b-28a9-48f7-98c0-be7b2fbd7efb None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:02:53.394368 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-65c9b89b-28a9-48f7-98c0-be7b2fbd7efb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:02:53.463421 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 510/1022] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 25 12:02:53 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 12:02:53.485839 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57fab62-ab11-413a-a821-3f2c67b0daeb None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:53.590456 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57fab62-ab11-413a-a821-3f2c67b0daeb None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:53.624518 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60dc6d6b-a1b9-41a2-94b0-80db2866f07f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:53.625775 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dc6d6b-a1b9-41a2-94b0-80db2866f07f None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:53.626063 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dc6d6b-a1b9-41a2-94b0-80db2866f07f None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:53.626348 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dc6d6b-a1b9-41a2-94b0-80db2866f07f None admin] PATH_INFO: `/v3/services/image` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:53.627008 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60dc6d6b-a1b9-41a2-94b0-80db2866f07f None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:53.628411 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60dc6d6b-a1b9-41a2-94b0-80db2866f07f None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:53.639987 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-60dc6d6b-a1b9-41a2-94b0-80db2866f07f None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 25 12:02:53.639987 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:02:53.639987 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 12:02:53.639987 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 12:02:53.639987 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 12:02:53.639987 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 12:02:53.639987 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 12:02:53.639987 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 12:02:53.639987 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 25 12:02:53.639987 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jul 25 12:02:53.639987 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 12:02:53.639987 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jul 25 12:02:53.639987 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:02:53.639987 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:02:53.639987 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:02:53.639987 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:02:53.639987 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:02:53.640998 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:02:53.640998 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:02:53.640998 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:02:53.640998 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:02:53.640998 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:02:53.640998 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:02:53.640998 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 12:02:53.640998 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 12:02:53.640998 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 12:02:53.640998 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 12:02:53.640998 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:02:53.640998 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:02:53.640998 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:02:53.640998 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:02:53.640998 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:02:53.640998 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:02:53.641840 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:02:53.641840 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jul 25 12:02:53.641840 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:02:53.641840 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:02:53.641840 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:02:53.641840 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:02:53.641840 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:02:53.641840 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:02:53.641840 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:02:53.641840 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:02:53.641840 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 12:02:53.641840 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 12:02:53.641840 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 25 12:02:53.641840 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jul 25 12:02:53.643633 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 513/1023] 10.222.0.42 () {56 vars in 1105 bytes} [Mon Jul 25 12:02:53 2022] GET /identity/v3/services/image => generated 86 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:02:53.649549 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c9b89b-28a9-48f7-98c0-be7b2fbd7efb None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:53.651240 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:53.715934 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ab847e-897b-45d3-b5f4-d469356b3dfc None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:53.716356 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ab847e-897b-45d3-b5f4-d469356b3dfc None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:53.716707 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ab847e-897b-45d3-b5f4-d469356b3dfc None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:53.718202 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 514/1024] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 25 12:02:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 12:02:53.724326 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53723c3-b9b1-40d7-b3e3-04b54335fc33 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:53.724742 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53723c3-b9b1-40d7-b3e3-04b54335fc33 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:53.725027 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53723c3-b9b1-40d7-b3e3-04b54335fc33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:53.743463 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-f53723c3-b9b1-40d7-b3e3-04b54335fc33 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:02:53.744695 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-f53723c3-b9b1-40d7-b3e3-04b54335fc33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:02:53.746983 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:53.758091 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c9b89b-28a9-48f7-98c0-be7b2fbd7efb None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:53.800904 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e521f8-ef2e-48e5-b7e4-d3dc1d608e7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:53.801577 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e521f8-ef2e-48e5-b7e4-d3dc1d608e7b None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:53.801866 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e521f8-ef2e-48e5-b7e4-d3dc1d608e7b None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:53.802148 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e521f8-ef2e-48e5-b7e4-d3dc1d608e7b None admin] PATH_INFO: `/v3/services` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:53.802860 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e521f8-ef2e-48e5-b7e4-d3dc1d608e7b None admin] RBAC: Adding query filter params (name=image) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:02:53.803171 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e521f8-ef2e-48e5-b7e4-d3dc1d608e7b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:53.804591 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e521f8-ef2e-48e5-b7e4-d3dc1d608e7b None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:53.809390 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 511/1025] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jul 25 12:02:53 2022] GET /identity/v3/services?name=image => generated 122 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:02:53.815312 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e521f8-ef2e-48e5-b7e4-d3dc1d608e7b None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:53.823819 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 515/1026] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 25 12:02:53 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 12:02:53.847092 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53723c3-b9b1-40d7-b3e3-04b54335fc33 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:53.884809 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:53.920357 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e521f8-ef2e-48e5-b7e4-d3dc1d608e7b None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:53.951553 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53723c3-b9b1-40d7-b3e3-04b54335fc33 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:53.955297 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5d3b53-a073-4a93-b0df-59de689f50c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:53.956190 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5d3b53-a073-4a93-b0df-59de689f50c5 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:53.956654 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5d3b53-a073-4a93-b0df-59de689f50c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:53.957132 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5d3b53-a073-4a93-b0df-59de689f50c5 None admin] PATH_INFO: `/v3/services` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:53.958012 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5d3b53-a073-4a93-b0df-59de689f50c5 None admin] RBAC: Adding query filter params (type=image) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:02:53.958495 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5d3b53-a073-4a93-b0df-59de689f50c5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:53.960570 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5d3b53-a073-4a93-b0df-59de689f50c5 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:53.972749 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 512/1027] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jul 25 12:02:53 2022] GET /identity/v3/services?type=image => generated 350 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 12:02:53.974183 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5d3b53-a073-4a93-b0df-59de689f50c5 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:53.990615 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0917a585-070d-4946-b603-96759c58c2e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:53.991478 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0917a585-070d-4946-b603-96759c58c2e1 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:53.991794 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0917a585-070d-4946-b603-96759c58c2e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:53.992076 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0917a585-070d-4946-b603-96759c58c2e1 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:54.000355 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:54.005546 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0917a585-070d-4946-b603-96759c58c2e1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 12:02:54.014807 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-0917a585-070d-4946-b603-96759c58c2e1 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 25 12:02:54.014807 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:02:54.014807 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:02:54.014807 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:02:54.014807 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:02:54.014807 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:02:54.014807 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:02:54.014807 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:02:54.014807 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:02:54.014807 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:02:54.014807 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:02:54.014807 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:02:54.014807 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 12:02:54.014807 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 12:02:54.014807 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 12:02:54.014807 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 12:02:54.016384 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 12:02:54.016384 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 12:02:54.016384 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 12:02:54.016384 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 12:02:54.016384 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:02:54.016384 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:02:54.016384 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:02:54.016384 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:02:54.016384 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:02:54.016384 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:02:54.016384 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:02:54.016384 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jul 25 12:02:54.016384 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:02:54.016384 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:02:54.016384 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:02:54.016384 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:02:54.017241 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:02:54.017241 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:02:54.017241 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:02:54.017241 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:02:54.017241 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 12:02:54.017241 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 12:02:54.017241 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 12:02:54.017241 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 12:02:54.017241 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 12:02:54.017241 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 12:02:54.017241 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 25 12:02:54.017241 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jul 25 12:02:54.017241 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 516/1028] 10.222.0.42 () {56 vars in 1120 bytes} [Mon Jul 25 12:02:53 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 12:02:54.022442 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0917a585-070d-4946-b603-96759c58c2e1 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:54.084189 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5d3b53-a073-4a93-b0df-59de689f50c5 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:54.116858 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:54.124532 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc315698-7770-40a0-b793-3cd69c04dc94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:54.125538 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc315698-7770-40a0-b793-3cd69c04dc94 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:54.126031 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc315698-7770-40a0-b793-3cd69c04dc94 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:54.126548 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc315698-7770-40a0-b793-3cd69c04dc94 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:54.127690 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc315698-7770-40a0-b793-3cd69c04dc94 None admin] RBAC: Adding query filter params (interface=public, service_id=3a9ee9ac89de49c481564fd212233a01, region_id=RegionOne) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:02:54.128175 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc315698-7770-40a0-b793-3cd69c04dc94 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:54.129951 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc315698-7770-40a0-b793-3cd69c04dc94 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:54.132767 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0917a585-070d-4946-b603-96759c58c2e1 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:54.135979 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 513/1029] 10.222.0.42 () {56 vars in 1251 bytes} [Mon Jul 25 12:02:53 2022] GET /identity/v3/endpoints?service_id=3a9ee9ac89de49c481564fd212233a01&interface=public®ion_id=RegionOne => generated 194 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 12:02:54.141771 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc315698-7770-40a0-b793-3cd69c04dc94 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:54.171458 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9485e497-b40c-4909-8212-0d4af534cbf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:54.172688 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9485e497-b40c-4909-8212-0d4af534cbf4 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:54.172980 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9485e497-b40c-4909-8212-0d4af534cbf4 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:54.173263 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9485e497-b40c-4909-8212-0d4af534cbf4 None admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:54.174016 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9485e497-b40c-4909-8212-0d4af534cbf4 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:02:54.174333 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9485e497-b40c-4909-8212-0d4af534cbf4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:54.175731 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9485e497-b40c-4909-8212-0d4af534cbf4 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:54.183884 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 517/1030] 10.222.0.42 () {56 vars in 1115 bytes} [Mon Jul 25 12:02:54 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:02:54.190084 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9485e497-b40c-4909-8212-0d4af534cbf4 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:54.248032 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc315698-7770-40a0-b793-3cd69c04dc94 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:54.254730 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:54.283903 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b717db4-200d-40d7-9af9-1f8464bfc79c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:54.284567 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b717db4-200d-40d7-9af9-1f8464bfc79c None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:54.284854 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b717db4-200d-40d7-9af9-1f8464bfc79c None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:54.285203 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b717db4-200d-40d7-9af9-1f8464bfc79c None admin] PATH_INFO: `/v3/services` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:54.285895 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b717db4-200d-40d7-9af9-1f8464bfc79c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:54.287256 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b717db4-200d-40d7-9af9-1f8464bfc79c None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:54.293992 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 514/1031] 10.222.0.42 () {56 vars in 1087 bytes} [Mon Jul 25 12:02:54 2022] GET /identity/v3/services => generated 1957 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 12:02:54.303841 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9485e497-b40c-4909-8212-0d4af534cbf4 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:54.336832 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4756bde0-c3f7-4d2a-a80f-7a472095de44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:54.337633 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4756bde0-c3f7-4d2a-a80f-7a472095de44 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:54.337921 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4756bde0-c3f7-4d2a-a80f-7a472095de44 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:54.338207 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4756bde0-c3f7-4d2a-a80f-7a472095de44 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:54.350775 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4756bde0-c3f7-4d2a-a80f-7a472095de44 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:54.352309 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4756bde0-c3f7-4d2a-a80f-7a472095de44 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:54.371024 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-4756bde0-c3f7-4d2a-a80f-7a472095de44 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:02:54.371024 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:02:54.371024 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:02:54.371024 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:02:54.371024 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:02:54.371024 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:02:54.371024 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:02:54.371024 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:02:54.371024 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:02:54.371024 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:02:54.371024 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:02:54.371024 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:02:54.371024 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 12:02:54.371024 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 12:02:54.371024 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 12:02:54.372166 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 12:02:54.372166 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:02:54.372166 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:02:54.372166 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:02:54.372166 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:02:54.372166 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:02:54.372166 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:02:54.372166 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:02:54.372166 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jul 25 12:02:54.372166 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:02:54.372166 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:02:54.372166 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:02:54.372166 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:02:54.372166 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:02:54.372166 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:02:54.372166 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:02:54.372166 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:02:54.373241 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 12:02:54.373241 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 12:02:54.373241 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:02:54.373241 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jul 25 12:02:54.375706 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 518/1032] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jul 25 12:02:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:02:54.378722 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b717db4-200d-40d7-9af9-1f8464bfc79c None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:54.451184 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:54.495773 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b717db4-200d-40d7-9af9-1f8464bfc79c None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:54.530667 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5e8605-7fec-430d-b70a-21ff6cfbd701 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:54.532216 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5e8605-7fec-430d-b70a-21ff6cfbd701 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:54.532647 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5e8605-7fec-430d-b70a-21ff6cfbd701 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:54.533071 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5e8605-7fec-430d-b70a-21ff6cfbd701 None admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:54.533937 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5e8605-7fec-430d-b70a-21ff6cfbd701 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:02:54.534389 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5e8605-7fec-430d-b70a-21ff6cfbd701 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:54.535936 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5e8605-7fec-430d-b70a-21ff6cfbd701 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:54.549219 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 515/1033] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jul 25 12:02:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:02:54.554504 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4756bde0-c3f7-4d2a-a80f-7a472095de44 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:54.655877 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4756bde0-c3f7-4d2a-a80f-7a472095de44 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:54.670185 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:54.697378 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bc2a87-d6bf-42fc-bdb9-3ad893662c00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:54.698459 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bc2a87-d6bf-42fc-bdb9-3ad893662c00 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:54.698897 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bc2a87-d6bf-42fc-bdb9-3ad893662c00 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:54.699333 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bc2a87-d6bf-42fc-bdb9-3ad893662c00 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:54.707746 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90bc2a87-d6bf-42fc-bdb9-3ad893662c00 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:54.709449 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90bc2a87-d6bf-42fc-bdb9-3ad893662c00 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:54.724158 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-90bc2a87-d6bf-42fc-bdb9-3ad893662c00 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 25 12:02:54.724158 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:02:54.724158 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:02:54.724158 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:02:54.724158 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:02:54.724158 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:02:54.724158 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:02:54.724158 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:02:54.724158 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:02:54.724158 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:02:54.724158 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:02:54.724158 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:02:54.724158 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 12:02:54.724158 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 12:02:54.724158 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 12:02:54.724158 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 12:02:54.725192 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:02:54.725192 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:02:54.725192 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 12:02:54.725192 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 12:02:54.725192 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 12:02:54.725192 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 12:02:54.725192 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:02:54.725192 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:02:54.725192 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:02:54.725192 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:02:54.725192 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:02:54.725192 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jul 25 12:02:54.725192 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:02:54.725192 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:02:54.725192 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:02:54.725192 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:02:54.725192 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:02:54.725192 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:02:54.726136 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:02:54.726136 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:02:54.726136 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 12:02:54.726136 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 12:02:54.726136 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 12:02:54.726136 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 12:02:54.726136 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 12:02:54.726136 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 12:02:54.726136 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 25 12:02:54.726136 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jul 25 12:02:54.727620 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 519/1034] 10.222.0.42 () {56 vars in 1093 bytes} [Mon Jul 25 12:02:54 2022] GET /identity/v3/users/nova => generated 82 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 12:02:54.731742 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5e8605-7fec-430d-b70a-21ff6cfbd701 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:54.845859 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5e8605-7fec-430d-b70a-21ff6cfbd701 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:54.866000 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:54.880864 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2e0a75-bdc4-4f9b-b0fa-aa44375aaa9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:54.881538 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2e0a75-bdc4-4f9b-b0fa-aa44375aaa9e None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:54.881824 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2e0a75-bdc4-4f9b-b0fa-aa44375aaa9e None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:54.882106 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2e0a75-bdc4-4f9b-b0fa-aa44375aaa9e None admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:54.882948 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2e0a75-bdc4-4f9b-b0fa-aa44375aaa9e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:02:54.883264 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2e0a75-bdc4-4f9b-b0fa-aa44375aaa9e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:54.884722 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2e0a75-bdc4-4f9b-b0fa-aa44375aaa9e None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:54.902686 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 516/1035] 10.222.0.42 () {56 vars in 1133 bytes} [Mon Jul 25 12:02:54 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:02:54.908767 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bc2a87-d6bf-42fc-bdb9-3ad893662c00 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:55.009571 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bc2a87-d6bf-42fc-bdb9-3ad893662c00 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:55.014296 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d33a00-074f-480f-ba6e-12dcce99fdc8 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:55.014592 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d33a00-074f-480f-ba6e-12dcce99fdc8 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:55.014877 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d33a00-074f-480f-ba6e-12dcce99fdc8 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:55.016959 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 517/1036] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 25 12:02:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:02:55.021561 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e29f78c-7cf7-4d2c-9602-d56f3d917e97 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:55.021853 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e29f78c-7cf7-4d2c-9602-d56f3d917e97 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:55.022137 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e29f78c-7cf7-4d2c-9602-d56f3d917e97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:55.040774 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-5e29f78c-7cf7-4d2c-9602-d56f3d917e97 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:02:55.041959 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-5e29f78c-7cf7-4d2c-9602-d56f3d917e97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:02:55.046678 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536df2eb-0ed2-496a-a1c0-eb251cd946db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:55.047528 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536df2eb-0ed2-496a-a1c0-eb251cd946db None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:55.047991 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536df2eb-0ed2-496a-a1c0-eb251cd946db None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:55.048420 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536df2eb-0ed2-496a-a1c0-eb251cd946db None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:55.095639 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:55.116137 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 518/1037] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 25 12:02:55 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 25 12:02:55.129002 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536df2eb-0ed2-496a-a1c0-eb251cd946db None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:55.130529 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536df2eb-0ed2-496a-a1c0-eb251cd946db None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:55.140713 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e29f78c-7cf7-4d2c-9602-d56f3d917e97 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:55.233318 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 520/1038] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Jul 25 12:02:54 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 25 12:02:55.239311 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536df2eb-0ed2-496a-a1c0-eb251cd946db None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:55.245154 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e29f78c-7cf7-4d2c-9602-d56f3d917e97 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:55.255829 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:55.283321 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6c4186-5cf1-4c0d-93ad-3b866af87de5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:55.284073 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6c4186-5cf1-4c0d-93ad-3b866af87de5 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:55.284366 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6c4186-5cf1-4c0d-93ad-3b866af87de5 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:55.284628 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6c4186-5cf1-4c0d-93ad-3b866af87de5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:55.285799 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6c4186-5cf1-4c0d-93ad-3b866af87de5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:55.287230 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6c4186-5cf1-4c0d-93ad-3b866af87de5 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:55.289055 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 519/1039] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jul 25 12:02:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 12:02:55.294593 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6c4186-5cf1-4c0d-93ad-3b866af87de5 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:55.310180 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:55.342404 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536df2eb-0ed2-496a-a1c0-eb251cd946db None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:55.378553 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45722df-bd66-4d3c-90db-24f1645557e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:55.379689 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45722df-bd66-4d3c-90db-24f1645557e0 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:55.379980 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45722df-bd66-4d3c-90db-24f1645557e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:55.380376 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45722df-bd66-4d3c-90db-24f1645557e0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:55.390658 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45722df-bd66-4d3c-90db-24f1645557e0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:55.392133 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45722df-bd66-4d3c-90db-24f1645557e0 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:55.401832 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6c4186-5cf1-4c0d-93ad-3b866af87de5 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:55.412981 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-c45722df-bd66-4d3c-90db-24f1645557e0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:02:55.412981 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:02:55.412981 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:02:55.412981 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:02:55.412981 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:02:55.412981 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:02:55.412981 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:02:55.412981 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:02:55.412981 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:02:55.412981 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:02:55.412981 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:02:55.412981 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:02:55.412981 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 12:02:55.412981 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 12:02:55.412981 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 12:02:55.413972 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 12:02:55.413972 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:02:55.413972 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:02:55.413972 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:02:55.413972 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:02:55.413972 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:02:55.413972 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:02:55.413972 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:02:55.413972 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jul 25 12:02:55.413972 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:02:55.413972 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:02:55.413972 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:02:55.413972 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:02:55.413972 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:02:55.413972 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:02:55.413972 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:02:55.413972 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:02:55.416145 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 12:02:55.416145 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 12:02:55.416145 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:02:55.416145 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jul 25 12:02:55.416145 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 521/1040] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jul 25 12:02:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 12:02:55.420346 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45722df-bd66-4d3c-90db-24f1645557e0 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:55.444803 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f616fc4e-c034-418e-ab6b-ef29dd2529e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:55.445482 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f616fc4e-c034-418e-ab6b-ef29dd2529e8 None admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:55.445776 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f616fc4e-c034-418e-ab6b-ef29dd2529e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:55.446064 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f616fc4e-c034-418e-ab6b-ef29dd2529e8 None admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:55.446776 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f616fc4e-c034-418e-ab6b-ef29dd2529e8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:55.448275 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f616fc4e-c034-418e-ab6b-ef29dd2529e8 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:55.471832 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 520/1041] 10.222.0.42 () {60 vars in 1132 bytes} [Mon Jul 25 12:02:55 2022] POST /identity/v3/users => generated 285 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 12:02:55.535754 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45722df-bd66-4d3c-90db-24f1645557e0 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:55.561842 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:55.574367 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a1b3ea-85e0-4174-a979-065c41f1a3f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:55.575054 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a1b3ea-85e0-4174-a979-065c41f1a3f0 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:55.575348 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a1b3ea-85e0-4174-a979-065c41f1a3f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:55.575670 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a1b3ea-85e0-4174-a979-065c41f1a3f0 None admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:55.576413 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a1b3ea-85e0-4174-a979-065c41f1a3f0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:02:55.576730 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a1b3ea-85e0-4174-a979-065c41f1a3f0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:55.578673 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a1b3ea-85e0-4174-a979-065c41f1a3f0 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:55.596410 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 522/1042] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jul 25 12:02:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 12:02:55.598368 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f616fc4e-c034-418e-ab6b-ef29dd2529e8 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:55.633020 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:55.717747 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f616fc4e-c034-418e-ab6b-ef29dd2529e8 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:55.752090 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c3103d-4fcb-45d4-8cf6-6f313bf6e229 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:55.752892 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c3103d-4fcb-45d4-8cf6-6f313bf6e229 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:55.753179 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c3103d-4fcb-45d4-8cf6-6f313bf6e229 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:55.753461 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c3103d-4fcb-45d4-8cf6-6f313bf6e229 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:55.760359 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c3103d-4fcb-45d4-8cf6-6f313bf6e229 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:55.761817 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c3103d-4fcb-45d4-8cf6-6f313bf6e229 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:55.779451 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-37c3103d-4fcb-45d4-8cf6-6f313bf6e229 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 12:02:55.779451 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:02:55.779451 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:02:55.779451 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:02:55.779451 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:02:55.779451 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:02:55.779451 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:02:55.779451 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:02:55.779451 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:02:55.779451 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:02:55.779451 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:02:55.779451 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:02:55.779451 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 12:02:55.779451 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 12:02:55.779451 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 12:02:55.780376 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 12:02:55.780376 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:02:55.780376 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:02:55.780376 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:02:55.780376 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:02:55.780376 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:02:55.780376 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:02:55.780376 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:02:55.780376 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 25 12:02:55.780376 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:02:55.780376 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:02:55.780376 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:02:55.780376 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:02:55.780376 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:02:55.780376 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:02:55.780376 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:02:55.780376 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:02:55.783160 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 12:02:55.783160 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 12:02:55.783160 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 12:02:55.783160 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 12:02:55.783160 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 12:02:55.783160 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 12:02:55.783160 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 12:02:55.783160 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 25 12:02:55.783160 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 521/1043] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Jul 25 12:02:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:02:55.787774 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a1b3ea-85e0-4174-a979-065c41f1a3f0 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:55.895072 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a1b3ea-85e0-4174-a979-065c41f1a3f0 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:55.929316 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cf9fa4-5eb6-4f71-bb32-1400498de187 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:55.929970 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cf9fa4-5eb6-4f71-bb32-1400498de187 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:55.930256 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cf9fa4-5eb6-4f71-bb32-1400498de187 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:55.930535 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cf9fa4-5eb6-4f71-bb32-1400498de187 None admin] PATH_INFO: `/v3/projects` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:55.931315 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44cf9fa4-5eb6-4f71-bb32-1400498de187 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:02:55.931655 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44cf9fa4-5eb6-4f71-bb32-1400498de187 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:55.933077 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44cf9fa4-5eb6-4f71-bb32-1400498de187 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:55.950936 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 523/1044] 10.222.0.42 () {56 vars in 1148 bytes} [Mon Jul 25 12:02:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:02:55.952502 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c3103d-4fcb-45d4-8cf6-6f313bf6e229 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:56.047723 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:56.060788 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c3103d-4fcb-45d4-8cf6-6f313bf6e229 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:56.099648 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe73d7e1-75b3-47a3-b302-32c33610d9fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:56.100309 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe73d7e1-75b3-47a3-b302-32c33610d9fd None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:56.100594 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe73d7e1-75b3-47a3-b302-32c33610d9fd None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:56.100873 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe73d7e1-75b3-47a3-b302-32c33610d9fd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:56.101709 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe73d7e1-75b3-47a3-b302-32c33610d9fd None admin] RBAC: Adding query filter params (role.id=3297f946c8f04c9f8a159c7e36bf41a5, scope.project.id=b9f62b8f37304b8a9a4bb54205347577, user.id=f67142a1193d47c7bd33d7040268233c) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:02:56.102027 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe73d7e1-75b3-47a3-b302-32c33610d9fd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:56.103461 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe73d7e1-75b3-47a3-b302-32c33610d9fd None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:56.115632 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 522/1045] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Jul 25 12:02:55 2022] GET /identity/v3/role_assignments?user.id=f67142a1193d47c7bd33d7040268233c&scope.project.id=b9f62b8f37304b8a9a4bb54205347577&role.id=3297f946c8f04c9f8a159c7e36bf41a5 => generated 259 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:02:56.320291 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c92bebc-dba1-43e5-8db7-8b5b97e37efe None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:56.320587 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c92bebc-dba1-43e5-8db7-8b5b97e37efe None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:56.320777 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c92bebc-dba1-43e5-8db7-8b5b97e37efe None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:56.322269 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 524/1046] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 25 12:02:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:02:56.327167 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c779de-9dc2-4729-bd31-a15ec9bada9a None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:56.327461 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c779de-9dc2-4729-bd31-a15ec9bada9a None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:56.327806 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c779de-9dc2-4729-bd31-a15ec9bada9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:56.346097 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-59c779de-9dc2-4729-bd31-a15ec9bada9a None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:02:56.347276 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-59c779de-9dc2-4729-bd31-a15ec9bada9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:02:56.412790 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 523/1047] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 25 12:02:56 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 12:02:56.440970 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c92bebc-dba1-43e5-8db7-8b5b97e37efe None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:56.491077 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:56.548257 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c92bebc-dba1-43e5-8db7-8b5b97e37efe None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:56.582931 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe8d092-a28e-4f08-84e0-6d0625dd630d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:56.583671 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe8d092-a28e-4f08-84e0-6d0625dd630d None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:56.583957 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe8d092-a28e-4f08-84e0-6d0625dd630d None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:56.584233 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe8d092-a28e-4f08-84e0-6d0625dd630d None admin] PATH_INFO: `/v3/services/image` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:56.584879 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe8d092-a28e-4f08-84e0-6d0625dd630d None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:56.586238 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe8d092-a28e-4f08-84e0-6d0625dd630d None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:56.597923 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-6fe8d092-a28e-4f08-84e0-6d0625dd630d None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 25 12:02:56.597923 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:02:56.597923 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 12:02:56.597923 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 12:02:56.597923 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 12:02:56.597923 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 12:02:56.597923 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 12:02:56.597923 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 12:02:56.597923 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 25 12:02:56.597923 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jul 25 12:02:56.597923 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 12:02:56.597923 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jul 25 12:02:56.597923 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:02:56.597923 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:02:56.597923 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:02:56.597923 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:02:56.597923 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:02:56.598908 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:02:56.598908 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:02:56.598908 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:02:56.598908 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:02:56.598908 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:02:56.598908 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:02:56.598908 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 12:02:56.598908 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 12:02:56.598908 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 12:02:56.598908 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 12:02:56.598908 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:02:56.598908 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:02:56.598908 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:02:56.598908 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:02:56.598908 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:02:56.598908 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:02:56.600174 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:02:56.600174 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jul 25 12:02:56.600174 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:02:56.600174 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:02:56.600174 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:02:56.600174 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:02:56.600174 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:02:56.600174 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:02:56.600174 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:02:56.600174 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:02:56.600174 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 12:02:56.600174 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 12:02:56.600174 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 25 12:02:56.600174 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jul 25 12:02:56.603861 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 525/1048] 10.222.0.42 () {56 vars in 1105 bytes} [Mon Jul 25 12:02:56 2022] GET /identity/v3/services/image => generated 86 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:02:56.605015 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c779de-9dc2-4729-bd31-a15ec9bada9a None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:56.691507 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:56.707110 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c779de-9dc2-4729-bd31-a15ec9bada9a None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:56.747872 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033a20a1-a28e-45a7-a0ae-a6765676aec0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:56.748992 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033a20a1-a28e-45a7-a0ae-a6765676aec0 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:56.749460 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033a20a1-a28e-45a7-a0ae-a6765676aec0 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:56.749953 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033a20a1-a28e-45a7-a0ae-a6765676aec0 None admin] PATH_INFO: `/v3/services` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:56.750900 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033a20a1-a28e-45a7-a0ae-a6765676aec0 None admin] RBAC: Adding query filter params (name=image) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:02:56.751210 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033a20a1-a28e-45a7-a0ae-a6765676aec0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:56.752613 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033a20a1-a28e-45a7-a0ae-a6765676aec0 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:56.758127 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 524/1049] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jul 25 12:02:56 2022] GET /identity/v3/services?name=image => generated 122 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:02:56.763636 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe8d092-a28e-4f08-84e0-6d0625dd630d None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:56.872224 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe8d092-a28e-4f08-84e0-6d0625dd630d None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:56.881523 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:56.908296 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75492ef2-3627-4491-9e4d-f23efe78cd48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:56.908968 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75492ef2-3627-4491-9e4d-f23efe78cd48 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:56.909261 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75492ef2-3627-4491-9e4d-f23efe78cd48 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:56.909597 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75492ef2-3627-4491-9e4d-f23efe78cd48 None admin] PATH_INFO: `/v3/services` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:56.910306 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75492ef2-3627-4491-9e4d-f23efe78cd48 None admin] RBAC: Adding query filter params (type=image) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:02:56.910670 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75492ef2-3627-4491-9e4d-f23efe78cd48 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:56.912071 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75492ef2-3627-4491-9e4d-f23efe78cd48 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:56.922434 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 526/1050] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jul 25 12:02:56 2022] GET /identity/v3/services?type=image => generated 350 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 12:02:56.923521 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033a20a1-a28e-45a7-a0ae-a6765676aec0 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:57.022906 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033a20a1-a28e-45a7-a0ae-a6765676aec0 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:57.057150 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc34edf4-ce91-41b2-869f-ad1333ea843d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:57.057815 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc34edf4-ce91-41b2-869f-ad1333ea843d None admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:57.058156 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc34edf4-ce91-41b2-869f-ad1333ea843d None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:57.058447 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc34edf4-ce91-41b2-869f-ad1333ea843d None admin] PATH_INFO: `/v3/endpoints` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:57.059203 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc34edf4-ce91-41b2-869f-ad1333ea843d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:57.060662 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc34edf4-ce91-41b2-869f-ad1333ea843d None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:57.095590 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 525/1051] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Jul 25 12:02:56 2022] POST /identity/v3/endpoints => generated 327 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 12:02:57.535383 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b0bc6c-34ff-4069-a5e5-66dc0e96fa21 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:57.535773 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b0bc6c-34ff-4069-a5e5-66dc0e96fa21 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:57.535993 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b0bc6c-34ff-4069-a5e5-66dc0e96fa21 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:57.537499 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 527/1052] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 25 12:02:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:02:57.542701 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50465c4-fdc2-48bc-8872-452416958844 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:57.543006 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50465c4-fdc2-48bc-8872-452416958844 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:57.543295 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50465c4-fdc2-48bc-8872-452416958844 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:57.557300 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:57.563701 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-a50465c4-fdc2-48bc-8872-452416958844 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:02:57.565227 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-a50465c4-fdc2-48bc-8872-452416958844 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:02:57.638285 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 526/1053] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 25 12:02:57 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 12:02:57.661019 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b0bc6c-34ff-4069-a5e5-66dc0e96fa21 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:57.764226 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b0bc6c-34ff-4069-a5e5-66dc0e96fa21 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:57.804519 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4083e58-2814-4d14-b7b1-b5e9a358d7d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:57.805322 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4083e58-2814-4d14-b7b1-b5e9a358d7d2 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:57.805612 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4083e58-2814-4d14-b7b1-b5e9a358d7d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:57.805891 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4083e58-2814-4d14-b7b1-b5e9a358d7d2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:57.816165 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4083e58-2814-4d14-b7b1-b5e9a358d7d2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 12:02:57.823736 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-e4083e58-2814-4d14-b7b1-b5e9a358d7d2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 12:02:57.823736 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:02:57.823736 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:02:57.823736 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:02:57.823736 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:02:57.823736 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:02:57.823736 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:02:57.823736 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:02:57.823736 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:02:57.823736 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:02:57.823736 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:02:57.823736 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:02:57.823736 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 12:02:57.823736 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 12:02:57.823736 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 12:02:57.823736 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 12:02:57.824667 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 12:02:57.824667 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 12:02:57.824667 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 12:02:57.824667 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 12:02:57.824667 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:02:57.824667 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:02:57.824667 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:02:57.824667 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:02:57.824667 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:02:57.824667 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:02:57.824667 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:02:57.824667 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jul 25 12:02:57.824667 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:02:57.824667 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:02:57.824667 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:02:57.824667 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:02:57.825601 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:02:57.825601 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:02:57.825601 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:02:57.825601 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:02:57.825601 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 12:02:57.825601 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 12:02:57.825601 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 12:02:57.825601 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 12:02:57.825601 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 12:02:57.825601 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 12:02:57.825601 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 12:02:57.825601 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jul 25 12:02:57.828350 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 528/1054] 10.222.0.42 () {56 vars in 1096 bytes} [Mon Jul 25 12:02:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:02:57.831385 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50465c4-fdc2-48bc-8872-452416958844 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:57.931382 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50465c4-fdc2-48bc-8872-452416958844 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:57.960333 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:57.966316 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f96d77-7a08-470f-a61a-5a8d46c684ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:57.966984 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f96d77-7a08-470f-a61a-5a8d46c684ea None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:57.967274 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f96d77-7a08-470f-a61a-5a8d46c684ea None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:57.967573 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f96d77-7a08-470f-a61a-5a8d46c684ea None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:57.968879 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f96d77-7a08-470f-a61a-5a8d46c684ea None admin] RBAC: Adding query filter params (name=admin) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:02:57.969194 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f96d77-7a08-470f-a61a-5a8d46c684ea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:57.970554 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f96d77-7a08-470f-a61a-5a8d46c684ea None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:57.983441 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 527/1055] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Jul 25 12:02:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:02:57.984748 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4083e58-2814-4d14-b7b1-b5e9a358d7d2 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:58.016937 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:58.087121 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4083e58-2814-4d14-b7b1-b5e9a358d7d2 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:58.120905 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3ff0f9-e287-4349-834b-c6bdc2f75bdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:58.121836 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3ff0f9-e287-4349-834b-c6bdc2f75bdd None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:58.122124 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3ff0f9-e287-4349-834b-c6bdc2f75bdd None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:58.122453 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3ff0f9-e287-4349-834b-c6bdc2f75bdd None admin] PATH_INFO: `/v3/users/09e1099cc2be4c10a4d99bf2e2f1d458` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:58.136258 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3ff0f9-e287-4349-834b-c6bdc2f75bdd None admin] RBAC: Authorizing `identity:get_user(user_id=09e1099cc2be4c10a4d99bf2e2f1d458)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:58.137730 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3ff0f9-e287-4349-834b-c6bdc2f75bdd None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:58.144659 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 529/1056] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Jul 25 12:02:57 2022] GET /identity/v3/users/09e1099cc2be4c10a4d99bf2e2f1d458 => generated 285 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:02:58.145711 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f96d77-7a08-470f-a61a-5a8d46c684ea None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:58.149920 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b389ca6-d18a-4e61-91a5-36d2ff58f208 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:58.150221 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b389ca6-d18a-4e61-91a5-36d2ff58f208 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:58.150504 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b389ca6-d18a-4e61-91a5-36d2ff58f208 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:58.151873 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 530/1057] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 25 12:02:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:02:58.157217 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cdcdc3-f5ef-467a-adb7-00f469da6e17 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:58.157507 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cdcdc3-f5ef-467a-adb7-00f469da6e17 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:58.157786 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cdcdc3-f5ef-467a-adb7-00f469da6e17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:58.175835 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-99cdcdc3-f5ef-467a-adb7-00f469da6e17 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:02:58.177028 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-99cdcdc3-f5ef-467a-adb7-00f469da6e17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:02:58.249104 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 531/1058] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 25 12:02:58 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 25 12:02:58.254307 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f96d77-7a08-470f-a61a-5a8d46c684ea None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:58.271891 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cdcdc3-f5ef-467a-adb7-00f469da6e17 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:58.290986 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8042a199-f14b-4661-b1ba-20c2c9c24b4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:58.291673 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8042a199-f14b-4661-b1ba-20c2c9c24b4d None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:58.291965 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8042a199-f14b-4661-b1ba-20c2c9c24b4d None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:58.292408 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:58.292935 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8042a199-f14b-4661-b1ba-20c2c9c24b4d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:58.357717 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:58.366467 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8042a199-f14b-4661-b1ba-20c2c9c24b4d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:58.368082 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8042a199-f14b-4661-b1ba-20c2c9c24b4d None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:58.372311 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cdcdc3-f5ef-467a-adb7-00f469da6e17 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:58.407063 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5dccca-b309-43c8-85ef-7d4c4e207153 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:58.408499 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5dccca-b309-43c8-85ef-7d4c4e207153 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:58.409006 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5dccca-b309-43c8-85ef-7d4c4e207153 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:58.409515 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5dccca-b309-43c8-85ef-7d4c4e207153 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:58.423668 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5dccca-b309-43c8-85ef-7d4c4e207153 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 12:02:58.433322 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-6e5dccca-b309-43c8-85ef-7d4c4e207153 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 25 12:02:58.433322 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:02:58.433322 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:02:58.433322 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:02:58.433322 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:02:58.433322 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:02:58.433322 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:02:58.433322 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:02:58.433322 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:02:58.433322 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:02:58.433322 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:02:58.433322 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:02:58.433322 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 12:02:58.433322 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 12:02:58.433322 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 12:02:58.433322 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 12:02:58.434281 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 12:02:58.434281 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 12:02:58.434281 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 12:02:58.434281 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 12:02:58.434281 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:02:58.434281 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:02:58.434281 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:02:58.434281 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:02:58.434281 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:02:58.434281 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:02:58.434281 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:02:58.434281 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jul 25 12:02:58.434281 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:02:58.434281 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:02:58.434281 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:02:58.434281 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:02:58.440437 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:02:58.440437 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:02:58.440437 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:02:58.440437 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:02:58.440437 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 12:02:58.440437 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 12:02:58.440437 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 12:02:58.440437 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 12:02:58.440437 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 12:02:58.440437 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 12:02:58.440437 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 25 12:02:58.440437 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jul 25 12:02:58.440437 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 532/1059] 10.222.0.42 () {56 vars in 1120 bytes} [Mon Jul 25 12:02:58 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 12:02:58.441126 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5dccca-b309-43c8-85ef-7d4c4e207153 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:58.483921 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 528/1060] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Jul 25 12:02:58 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 25 12:02:58.487407 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8042a199-f14b-4661-b1ba-20c2c9c24b4d None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:58.524792 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:58.547342 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5dccca-b309-43c8-85ef-7d4c4e207153 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:58.583817 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b10e208-5495-4aee-8d9b-434921844ec3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:58.584601 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b10e208-5495-4aee-8d9b-434921844ec3 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:58.585014 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b10e208-5495-4aee-8d9b-434921844ec3 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:58.585301 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b10e208-5495-4aee-8d9b-434921844ec3 None admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:58.586050 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b10e208-5495-4aee-8d9b-434921844ec3 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:02:58.586363 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b10e208-5495-4aee-8d9b-434921844ec3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:58.587419 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:58.588499 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b10e208-5495-4aee-8d9b-434921844ec3 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:58.595538 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8042a199-f14b-4661-b1ba-20c2c9c24b4d None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:58.596661 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 533/1061] 10.222.0.42 () {56 vars in 1115 bytes} [Mon Jul 25 12:02:58 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 12:02:58.602018 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b10e208-5495-4aee-8d9b-434921844ec3 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:58.632568 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2dabf3b-fc97-4bf3-b67b-b93caae7f2d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:58.633354 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dabf3b-fc97-4bf3-b67b-b93caae7f2d3 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:58.633641 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dabf3b-fc97-4bf3-b67b-b93caae7f2d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:58.633917 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dabf3b-fc97-4bf3-b67b-b93caae7f2d3 None admin] PATH_INFO: `/v3/projects/bff2d22cbce449d9ac721dd3cfd8b074` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:58.635328 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2dabf3b-fc97-4bf3-b67b-b93caae7f2d3 None admin] RBAC: Authorizing `identity:get_project(project_id=bff2d22cbce449d9ac721dd3cfd8b074)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:58.636800 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2dabf3b-fc97-4bf3-b67b-b93caae7f2d3 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:58.638893 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 529/1062] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Jul 25 12:02:58 2022] GET /identity/v3/projects/bff2d22cbce449d9ac721dd3cfd8b074 => generated 311 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 12:02:58.644636 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2dabf3b-fc97-4bf3-b67b-b93caae7f2d3 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:58.711599 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b10e208-5495-4aee-8d9b-434921844ec3 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:58.750882 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2dabf3b-fc97-4bf3-b67b-b93caae7f2d3 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:58.752590 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3309e93d-99e5-4089-8d4b-d0193fa25d48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:58.753542 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3309e93d-99e5-4089-8d4b-d0193fa25d48 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:58.753911 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3309e93d-99e5-4089-8d4b-d0193fa25d48 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:58.754194 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3309e93d-99e5-4089-8d4b-d0193fa25d48 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:58.765492 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3309e93d-99e5-4089-8d4b-d0193fa25d48 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:58.766935 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3309e93d-99e5-4089-8d4b-d0193fa25d48 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:58.786964 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:58.789436 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-3309e93d-99e5-4089-8d4b-d0193fa25d48 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:02:58.789436 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:02:58.789436 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:02:58.789436 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:02:58.789436 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:02:58.789436 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:02:58.789436 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:02:58.789436 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:02:58.789436 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:02:58.789436 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:02:58.789436 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:02:58.789436 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:02:58.789436 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 12:02:58.789436 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 12:02:58.789436 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 12:02:58.790998 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 12:02:58.790998 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:02:58.790998 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:02:58.790998 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:02:58.790998 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:02:58.790998 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:02:58.790998 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:02:58.790998 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:02:58.790998 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jul 25 12:02:58.790998 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:02:58.790998 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:02:58.790998 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:02:58.790998 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:02:58.790998 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:02:58.790998 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:02:58.790998 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:02:58.790998 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:02:58.792025 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 12:02:58.792025 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 12:02:58.792025 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:02:58.792025 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jul 25 12:02:58.793859 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 534/1063] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jul 25 12:02:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 12:02:58.795899 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ceeb9f-fb3f-4d63-ab5f-706bd8437c0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:58.797177 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ceeb9f-fb3f-4d63-ab5f-706bd8437c0a None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:58.797529 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ceeb9f-fb3f-4d63-ab5f-706bd8437c0a None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:58.797881 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ceeb9f-fb3f-4d63-ab5f-706bd8437c0a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:58.798801 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ceeb9f-fb3f-4d63-ab5f-706bd8437c0a None admin] RBAC: Adding query filter params (role.id=a7d20885b6d94928a34b3cc5b3a00006, scope.project.id=bff2d22cbce449d9ac721dd3cfd8b074, user.id=09e1099cc2be4c10a4d99bf2e2f1d458) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:02:58.799122 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ceeb9f-fb3f-4d63-ab5f-706bd8437c0a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:58.799654 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3309e93d-99e5-4089-8d4b-d0193fa25d48 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:58.800600 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ceeb9f-fb3f-4d63-ab5f-706bd8437c0a None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:58.810735 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 530/1064] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Jul 25 12:02:58 2022] GET /identity/v3/role_assignments?user.id=09e1099cc2be4c10a4d99bf2e2f1d458&scope.project.id=bff2d22cbce449d9ac721dd3cfd8b074&role.id=a7d20885b6d94928a34b3cc5b3a00006 => generated 259 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 12:02:58.912168 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3309e93d-99e5-4089-8d4b-d0193fa25d48 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:58.948608 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e0d471-a2ba-46c0-9179-bdc5213c4316 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:58.949360 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e0d471-a2ba-46c0-9179-bdc5213c4316 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:58.949652 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e0d471-a2ba-46c0-9179-bdc5213c4316 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:58.949989 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e0d471-a2ba-46c0-9179-bdc5213c4316 None admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:58.950734 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e0d471-a2ba-46c0-9179-bdc5213c4316 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:02:58.951052 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e0d471-a2ba-46c0-9179-bdc5213c4316 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:58.952454 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e0d471-a2ba-46c0-9179-bdc5213c4316 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:58.973073 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 535/1065] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jul 25 12:02:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 12:02:58.978575 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ceeb9f-fb3f-4d63-ab5f-706bd8437c0a None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:59.007956 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:59.083842 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ceeb9f-fb3f-4d63-ab5f-706bd8437c0a None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:59.117493 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fc795f-97e4-44a8-8525-1c8f3223516d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:59.118569 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fc795f-97e4-44a8-8525-1c8f3223516d None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:59.119016 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fc795f-97e4-44a8-8525-1c8f3223516d None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:59.119445 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fc795f-97e4-44a8-8525-1c8f3223516d None admin] PATH_INFO: `/v3/users/nova` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:59.127371 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fc795f-97e4-44a8-8525-1c8f3223516d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:59.129022 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fc795f-97e4-44a8-8525-1c8f3223516d None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:59.144043 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-20fc795f-97e4-44a8-8525-1c8f3223516d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 25 12:02:59.144043 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:02:59.144043 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:02:59.144043 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:02:59.144043 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:02:59.144043 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:02:59.144043 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:02:59.144043 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:02:59.144043 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:02:59.144043 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:02:59.144043 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:02:59.144043 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:02:59.144043 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 12:02:59.144043 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 12:02:59.144043 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 12:02:59.144043 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 12:02:59.145193 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:02:59.145193 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:02:59.145193 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 12:02:59.145193 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 12:02:59.145193 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 12:02:59.145193 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 12:02:59.145193 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:02:59.145193 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:02:59.145193 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:02:59.145193 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:02:59.145193 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:02:59.145193 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 25 12:02:59.145193 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:02:59.145193 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:02:59.145193 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:02:59.145193 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:02:59.145193 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:02:59.145193 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:02:59.146357 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:02:59.146357 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:02:59.146357 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 12:02:59.146357 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 12:02:59.146357 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 12:02:59.146357 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 12:02:59.146357 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 12:02:59.146357 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 12:02:59.146357 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 25 12:02:59.146357 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 25 12:02:59.148616 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 531/1066] 10.222.0.42 () {56 vars in 1093 bytes} [Mon Jul 25 12:02:58 2022] GET /identity/v3/users/nova => generated 82 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:02:59.150138 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b36705-3d8f-46fd-a56b-3211ff0abd24 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:59.150584 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b36705-3d8f-46fd-a56b-3211ff0abd24 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:59.150767 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b36705-3d8f-46fd-a56b-3211ff0abd24 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:59.152380 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 536/1067] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 25 12:02:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:02:59.153821 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fc795f-97e4-44a8-8525-1c8f3223516d None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:59.157399 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5679ff26-1e26-49f4-a01b-db53cfc33013 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:59.157879 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5679ff26-1e26-49f4-a01b-db53cfc33013 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:59.158163 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5679ff26-1e26-49f4-a01b-db53cfc33013 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:59.178102 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-5679ff26-1e26-49f4-a01b-db53cfc33013 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:02:59.179536 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-5679ff26-1e26-49f4-a01b-db53cfc33013 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:02:59.236053 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:59.247178 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:59.255477 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 537/1068] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 25 12:02:59 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 25 12:02:59.270857 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fc795f-97e4-44a8-8525-1c8f3223516d None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:59.277612 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5679ff26-1e26-49f4-a01b-db53cfc33013 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:59.307145 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f73fd68-1ad7-4b6b-9d66-68f49c6eadd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:59.307860 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f73fd68-1ad7-4b6b-9d66-68f49c6eadd7 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:59.308155 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f73fd68-1ad7-4b6b-9d66-68f49c6eadd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:59.308441 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f73fd68-1ad7-4b6b-9d66-68f49c6eadd7 None admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:59.309285 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f73fd68-1ad7-4b6b-9d66-68f49c6eadd7 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:02:59.309605 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f73fd68-1ad7-4b6b-9d66-68f49c6eadd7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:59.311038 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f73fd68-1ad7-4b6b-9d66-68f49c6eadd7 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:59.333661 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 532/1069] 10.222.0.42 () {56 vars in 1133 bytes} [Mon Jul 25 12:02:59 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:02:59.334868 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f73fd68-1ad7-4b6b-9d66-68f49c6eadd7 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:59.383117 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5679ff26-1e26-49f4-a01b-db53cfc33013 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:59.419593 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddeaeec-07e2-4624-8007-4aa7c2313d8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:59.420247 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddeaeec-07e2-4624-8007-4aa7c2313d8f None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:59.420650 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddeaeec-07e2-4624-8007-4aa7c2313d8f None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:59.420933 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddeaeec-07e2-4624-8007-4aa7c2313d8f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:59.442213 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f73fd68-1ad7-4b6b-9d66-68f49c6eadd7 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:59.476312 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:59.478763 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3938e0-aec4-45a6-a3d8-8de432852ee8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:59.479618 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3938e0-aec4-45a6-a3d8-8de432852ee8 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:59.479916 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3938e0-aec4-45a6-a3d8-8de432852ee8 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:59.480201 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3938e0-aec4-45a6-a3d8-8de432852ee8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:59.491259 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3938e0-aec4-45a6-a3d8-8de432852ee8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:59.492739 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3938e0-aec4-45a6-a3d8-8de432852ee8 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:59.495342 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:59.496026 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bddeaeec-07e2-4624-8007-4aa7c2313d8f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:59.497609 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bddeaeec-07e2-4624-8007-4aa7c2313d8f None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:59.513164 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-0c3938e0-aec4-45a6-a3d8-8de432852ee8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:02:59.513164 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:02:59.513164 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:02:59.513164 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:02:59.513164 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:02:59.513164 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:02:59.513164 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:02:59.513164 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:02:59.513164 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:02:59.513164 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:02:59.513164 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:02:59.513164 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:02:59.513164 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 12:02:59.513164 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 12:02:59.513164 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 12:02:59.514273 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 12:02:59.514273 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:02:59.514273 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:02:59.514273 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:02:59.514273 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:02:59.514273 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:02:59.514273 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:02:59.514273 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:02:59.514273 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 25 12:02:59.514273 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:02:59.514273 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:02:59.514273 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:02:59.514273 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:02:59.514273 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:02:59.514273 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:02:59.514273 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:02:59.514273 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:02:59.515410 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 12:02:59.515410 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 12:02:59.515410 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:02:59.515410 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 25 12:02:59.521982 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 533/1070] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jul 25 12:02:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 12:02:59.523019 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3938e0-aec4-45a6-a3d8-8de432852ee8 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:59.611045 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 538/1071] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Jul 25 12:02:59 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 25 12:02:59.616658 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddeaeec-07e2-4624-8007-4aa7c2313d8f None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:59.634808 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3938e0-aec4-45a6-a3d8-8de432852ee8 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:59.670315 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12df4b9-7beb-4314-b9b3-c7c80e9c877e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:59.670984 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12df4b9-7beb-4314-b9b3-c7c80e9c877e None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:59.671926 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12df4b9-7beb-4314-b9b3-c7c80e9c877e None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:59.672211 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12df4b9-7beb-4314-b9b3-c7c80e9c877e None admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:59.672939 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12df4b9-7beb-4314-b9b3-c7c80e9c877e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:02:59.673255 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12df4b9-7beb-4314-b9b3-c7c80e9c877e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:59.674625 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12df4b9-7beb-4314-b9b3-c7c80e9c877e None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:59.686293 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 534/1072] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jul 25 12:02:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:02:59.691772 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12df4b9-7beb-4314-b9b3-c7c80e9c877e None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:59.705983 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:59.724530 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddeaeec-07e2-4624-8007-4aa7c2313d8f None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:59.750314 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:59.761014 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b923f1e-ec83-49ee-aa06-c18970a3b9c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:59.761811 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b923f1e-ec83-49ee-aa06-c18970a3b9c5 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:59.762100 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b923f1e-ec83-49ee-aa06-c18970a3b9c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:59.762386 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b923f1e-ec83-49ee-aa06-c18970a3b9c5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:59.777473 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b923f1e-ec83-49ee-aa06-c18970a3b9c5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:59.777473 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b923f1e-ec83-49ee-aa06-c18970a3b9c5 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:59.796098 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-8b923f1e-ec83-49ee-aa06-c18970a3b9c5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:02:59.796098 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:02:59.796098 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:02:59.796098 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:02:59.796098 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:02:59.796098 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:02:59.796098 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:02:59.796098 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:02:59.796098 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:02:59.796098 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:02:59.796098 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:02:59.796098 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:02:59.796098 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 12:02:59.796098 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 12:02:59.796098 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 12:02:59.797570 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 12:02:59.797570 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:02:59.797570 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:02:59.797570 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:02:59.797570 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:02:59.797570 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:02:59.797570 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:02:59.797570 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:02:59.797570 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jul 25 12:02:59.797570 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:02:59.797570 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:02:59.797570 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:02:59.797570 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:02:59.797570 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:02:59.797570 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:02:59.797570 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:02:59.797570 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:02:59.798466 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 12:02:59.798466 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 12:02:59.798466 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:02:59.798466 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jul 25 12:02:59.798466 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 539/1073] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jul 25 12:02:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 12:02:59.803987 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b923f1e-ec83-49ee-aa06-c18970a3b9c5 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:59.805329 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12df4b9-7beb-4314-b9b3-c7c80e9c877e None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:59.841634 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aeefc7b-25d5-4595-8726-cd427aaf1452 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:59.842436 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aeefc7b-25d5-4595-8726-cd427aaf1452 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:59.842724 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aeefc7b-25d5-4595-8726-cd427aaf1452 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:59.843008 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aeefc7b-25d5-4595-8726-cd427aaf1452 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:59.851530 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aeefc7b-25d5-4595-8726-cd427aaf1452 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:59.853023 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aeefc7b-25d5-4595-8726-cd427aaf1452 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:59.868295 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-3aeefc7b-25d5-4595-8726-cd427aaf1452 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 12:02:59.868295 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:02:59.868295 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:02:59.868295 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:02:59.868295 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:02:59.868295 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:02:59.868295 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:02:59.868295 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:02:59.868295 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:02:59.868295 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:02:59.868295 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:02:59.868295 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:02:59.868295 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 12:02:59.868295 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 12:02:59.868295 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 12:02:59.869252 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 12:02:59.869252 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:02:59.869252 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:02:59.869252 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:02:59.869252 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:02:59.869252 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:02:59.869252 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:02:59.869252 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:02:59.869252 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 25 12:02:59.869252 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:02:59.869252 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:02:59.869252 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:02:59.869252 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:02:59.869252 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:02:59.869252 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:02:59.869252 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:02:59.869252 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:02:59.870360 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 12:02:59.870360 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 12:02:59.870360 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 12:02:59.870360 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 12:02:59.870360 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 12:02:59.870360 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 12:02:59.870360 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 12:02:59.870360 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 25 12:02:59.870360 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 535/1074] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Jul 25 12:02:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:02:59.875120 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aeefc7b-25d5-4595-8726-cd427aaf1452 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:02:59.917176 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b923f1e-ec83-49ee-aa06-c18970a3b9c5 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:02:59.953856 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9b96cd-de31-4b83-9d61-130e3bc6cc86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:02:59.954668 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9b96cd-de31-4b83-9d61-130e3bc6cc86 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:02:59.954959 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9b96cd-de31-4b83-9d61-130e3bc6cc86 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:02:59.955246 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9b96cd-de31-4b83-9d61-130e3bc6cc86 None admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:02:59.956191 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9b96cd-de31-4b83-9d61-130e3bc6cc86 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:02:59.956459 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9b96cd-de31-4b83-9d61-130e3bc6cc86 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:02:59.957942 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9b96cd-de31-4b83-9d61-130e3bc6cc86 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:02:59.960484 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:02:59.975756 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 540/1075] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jul 25 12:02:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 12:02:59.991673 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aeefc7b-25d5-4595-8726-cd427aaf1452 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:03:00.011504 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:03:00.032544 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad78f82c-4b0e-48f1-b62b-23e55717d1cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:03:00.033230 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad78f82c-4b0e-48f1-b62b-23e55717d1cb None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:03:00.033517 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad78f82c-4b0e-48f1-b62b-23e55717d1cb None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:03:00.033798 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad78f82c-4b0e-48f1-b62b-23e55717d1cb None admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:03:00.034613 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad78f82c-4b0e-48f1-b62b-23e55717d1cb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:03:00.034927 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad78f82c-4b0e-48f1-b62b-23e55717d1cb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:03:00.036413 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad78f82c-4b0e-48f1-b62b-23e55717d1cb None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:03:00.054723 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 536/1076] 10.222.0.42 () {56 vars in 1148 bytes} [Mon Jul 25 12:02:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 12:03:00.056105 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9b96cd-de31-4b83-9d61-130e3bc6cc86 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:03:00.164823 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9b96cd-de31-4b83-9d61-130e3bc6cc86 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:03:00.203178 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8ab669-7650-44e6-a468-f479e7b38f72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:03:00.204108 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8ab669-7650-44e6-a468-f479e7b38f72 None admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:03:00.204399 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8ab669-7650-44e6-a468-f479e7b38f72 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:03:00.204682 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8ab669-7650-44e6-a468-f479e7b38f72 None admin] PATH_INFO: `/v3/projects/b9f62b8f37304b8a9a4bb54205347577/users/f67142a1193d47c7bd33d7040268233c/roles/3297f946c8f04c9f8a159c7e36bf41a5` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:03:00.207209 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8ab669-7650-44e6-a468-f479e7b38f72 None admin] RBAC: Authorizing `identity:create_grant(project_id=b9f62b8f37304b8a9a4bb54205347577, user_id=f67142a1193d47c7bd33d7040268233c, role_id=3297f946c8f04c9f8a159c7e36bf41a5)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:03:00.208732 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8ab669-7650-44e6-a468-f479e7b38f72 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:03:00.225018 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 541/1077] 10.222.0.42 () {58 vars in 1439 bytes} [Mon Jul 25 12:03:00 2022] PUT /identity/v3/projects/b9f62b8f37304b8a9a4bb54205347577/users/f67142a1193d47c7bd33d7040268233c/roles/3297f946c8f04c9f8a159c7e36bf41a5 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:03:00.822041 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79265dd-c76e-4295-a26f-582975127a1e None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:03:00.822335 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79265dd-c76e-4295-a26f-582975127a1e None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:03:00.822621 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79265dd-c76e-4295-a26f-582975127a1e None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:03:00.824055 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 537/1078] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 25 12:03:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:03:00.829429 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ec10e1-99f5-4002-95ca-ed49a32c4c4b None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:03:00.829724 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ec10e1-99f5-4002-95ca-ed49a32c4c4b None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:03:00.830007 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ec10e1-99f5-4002-95ca-ed49a32c4c4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:03:00.848001 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-e3ec10e1-99f5-4002-95ca-ed49a32c4c4b None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:03:00.849188 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-e3ec10e1-99f5-4002-95ca-ed49a32c4c4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:03:00.899608 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:03:00.917724 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 542/1079] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 25 12:03:00 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 12:03:00.940763 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79265dd-c76e-4295-a26f-582975127a1e None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:03:01.039053 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79265dd-c76e-4295-a26f-582975127a1e None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:03:01.071789 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207a7b9a-f131-4c9c-8149-b321a1adc8c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:03:01.072641 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207a7b9a-f131-4c9c-8149-b321a1adc8c9 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:03:01.072861 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207a7b9a-f131-4c9c-8149-b321a1adc8c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:03:01.073141 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207a7b9a-f131-4c9c-8149-b321a1adc8c9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:03:01.083569 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207a7b9a-f131-4c9c-8149-b321a1adc8c9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 12:03:01.091039 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-207a7b9a-f131-4c9c-8149-b321a1adc8c9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 12:03:01.091039 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:03:01.091039 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:03:01.091039 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:03:01.091039 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:03:01.091039 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:03:01.091039 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:03:01.091039 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:03:01.091039 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:03:01.091039 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:03:01.091039 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:03:01.091039 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:03:01.091039 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 12:03:01.091039 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 12:03:01.091039 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 12:03:01.091039 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 12:03:01.092248 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 12:03:01.092248 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 12:03:01.092248 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 12:03:01.092248 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 12:03:01.092248 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:03:01.092248 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:03:01.092248 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:03:01.092248 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:03:01.092248 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:03:01.092248 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:03:01.092248 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:03:01.092248 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 25 12:03:01.092248 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:03:01.092248 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:03:01.092248 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:03:01.092248 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:03:01.093150 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:03:01.093150 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:03:01.093150 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:03:01.093150 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:03:01.093150 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 12:03:01.093150 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 12:03:01.093150 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 12:03:01.093150 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 12:03:01.093150 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 12:03:01.093150 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 12:03:01.093150 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 12:03:01.093150 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 25 12:03:01.093150 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 538/1080] 10.222.0.42 () {56 vars in 1096 bytes} [Mon Jul 25 12:03:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:03:01.098235 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ec10e1-99f5-4002-95ca-ed49a32c4c4b None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:03:01.204500 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ec10e1-99f5-4002-95ca-ed49a32c4c4b None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:03:01.237923 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f81a92-d8b7-483e-bc8e-4c9af547be3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:03:01.238591 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f81a92-d8b7-483e-bc8e-4c9af547be3a None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:03:01.238884 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f81a92-d8b7-483e-bc8e-4c9af547be3a None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:03:01.239172 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f81a92-d8b7-483e-bc8e-4c9af547be3a None admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:03:01.239959 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f81a92-d8b7-483e-bc8e-4c9af547be3a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:03:01.240274 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f81a92-d8b7-483e-bc8e-4c9af547be3a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:03:01.241639 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f81a92-d8b7-483e-bc8e-4c9af547be3a None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:03:01.250149 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 543/1081] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Jul 25 12:03:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:03:01.255905 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207a7b9a-f131-4c9c-8149-b321a1adc8c9 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:03:01.290907 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:03:01.357866 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207a7b9a-f131-4c9c-8149-b321a1adc8c9 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:03:01.391560 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc74893-df82-4c2f-a973-3102a7af272d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:03:01.392431 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc74893-df82-4c2f-a973-3102a7af272d None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:03:01.392720 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc74893-df82-4c2f-a973-3102a7af272d None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:03:01.393003 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc74893-df82-4c2f-a973-3102a7af272d None admin] PATH_INFO: `/v3/users/09e1099cc2be4c10a4d99bf2e2f1d458` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:03:01.394196 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc74893-df82-4c2f-a973-3102a7af272d None admin] RBAC: Authorizing `identity:get_user(user_id=09e1099cc2be4c10a4d99bf2e2f1d458)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:03:01.395658 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc74893-df82-4c2f-a973-3102a7af272d None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:03:01.397294 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 539/1082] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Jul 25 12:03:01 2022] GET /identity/v3/users/09e1099cc2be4c10a4d99bf2e2f1d458 => generated 285 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:03:01.403334 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f81a92-d8b7-483e-bc8e-4c9af547be3a None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:03:01.452275 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:03:01.502763 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f81a92-d8b7-483e-bc8e-4c9af547be3a None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:03:01.535755 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a4e460-eda8-4ddc-9775-9b48a99d8650 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:03:01.536605 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a4e460-eda8-4ddc-9775-9b48a99d8650 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:03:01.536889 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a4e460-eda8-4ddc-9775-9b48a99d8650 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:03:01.537169 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a4e460-eda8-4ddc-9775-9b48a99d8650 None admin] PATH_INFO: `/v3/projects/bff2d22cbce449d9ac721dd3cfd8b074` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:03:01.538331 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a4e460-eda8-4ddc-9775-9b48a99d8650 None admin] RBAC: Authorizing `identity:get_project(project_id=bff2d22cbce449d9ac721dd3cfd8b074)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:03:01.539817 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a4e460-eda8-4ddc-9775-9b48a99d8650 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:03:01.541811 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 544/1083] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Jul 25 12:03:01 2022] GET /identity/v3/projects/bff2d22cbce449d9ac721dd3cfd8b074 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:03:01.548561 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc74893-df82-4c2f-a973-3102a7af272d None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:03:01.644685 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc74893-df82-4c2f-a973-3102a7af272d None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:03:01.646990 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:03:01.679516 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035d389a-66e4-4c74-bfdb-ccf983c88c26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:03:01.680431 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035d389a-66e4-4c74-bfdb-ccf983c88c26 None admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:03:01.680720 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035d389a-66e4-4c74-bfdb-ccf983c88c26 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:03:01.681002 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035d389a-66e4-4c74-bfdb-ccf983c88c26 None admin] PATH_INFO: `/v3/projects/bff2d22cbce449d9ac721dd3cfd8b074/users/09e1099cc2be4c10a4d99bf2e2f1d458/roles/a7d20885b6d94928a34b3cc5b3a00006` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:03:01.682968 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035d389a-66e4-4c74-bfdb-ccf983c88c26 None admin] RBAC: Authorizing `identity:create_grant(project_id=bff2d22cbce449d9ac721dd3cfd8b074, user_id=09e1099cc2be4c10a4d99bf2e2f1d458, role_id=a7d20885b6d94928a34b3cc5b3a00006)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:03:01.684446 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035d389a-66e4-4c74-bfdb-ccf983c88c26 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:03:01.699158 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 540/1084] 10.222.0.42 () {58 vars in 1439 bytes} [Mon Jul 25 12:03:01 2022] PUT /identity/v3/projects/bff2d22cbce449d9ac721dd3cfd8b074/users/09e1099cc2be4c10a4d99bf2e2f1d458/roles/a7d20885b6d94928a34b3cc5b3a00006 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 12:03:02.204710 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba4bedd-9ceb-4ffa-bcf5-e5fb9f098c75 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:03:02.205298 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba4bedd-9ceb-4ffa-bcf5-e5fb9f098c75 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:03:02.205298 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba4bedd-9ceb-4ffa-bcf5-e5fb9f098c75 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:03:02.206628 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 545/1085] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 25 12:03:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 12:03:02.211761 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42638257-4bc0-4988-9077-2f9dd97828a0 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:03:02.212145 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42638257-4bc0-4988-9077-2f9dd97828a0 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:03:02.212326 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42638257-4bc0-4988-9077-2f9dd97828a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:03:02.231301 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-42638257-4bc0-4988-9077-2f9dd97828a0 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:03:02.232534 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-42638257-4bc0-4988-9077-2f9dd97828a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:03:02.298318 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 541/1086] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 25 12:03:02 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 12:03:02.321116 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba4bedd-9ceb-4ffa-bcf5-e5fb9f098c75 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:03:02.422626 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba4bedd-9ceb-4ffa-bcf5-e5fb9f098c75 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:03:02.438571 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:03:02.456817 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351180ea-9f3a-4844-8f2f-a6ed85b7cd74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:03:02.457571 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351180ea-9f3a-4844-8f2f-a6ed85b7cd74 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:03:02.457862 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351180ea-9f3a-4844-8f2f-a6ed85b7cd74 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:03:02.458147 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351180ea-9f3a-4844-8f2f-a6ed85b7cd74 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:03:02.468832 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351180ea-9f3a-4844-8f2f-a6ed85b7cd74 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 12:03:02.477010 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-351180ea-9f3a-4844-8f2f-a6ed85b7cd74 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 25 12:03:02.477010 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:03:02.477010 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:03:02.477010 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:03:02.477010 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:03:02.477010 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:03:02.477010 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:03:02.477010 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:03:02.477010 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:03:02.477010 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:03:02.477010 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:03:02.477010 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:03:02.477010 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 12:03:02.477010 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 12:03:02.477010 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 12:03:02.477010 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 12:03:02.477987 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 12:03:02.477987 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 12:03:02.477987 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 12:03:02.477987 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 12:03:02.477987 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:03:02.477987 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:03:02.477987 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:03:02.477987 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:03:02.477987 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:03:02.477987 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:03:02.477987 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:03:02.477987 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jul 25 12:03:02.477987 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:03:02.477987 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:03:02.477987 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:03:02.477987 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:03:02.478816 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:03:02.478816 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:03:02.478816 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:03:02.478816 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:03:02.478816 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 12:03:02.478816 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 12:03:02.478816 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 12:03:02.478816 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 12:03:02.478816 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 12:03:02.478816 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 12:03:02.478816 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 25 12:03:02.478816 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jul 25 12:03:02.478816 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 546/1087] 10.222.0.42 () {56 vars in 1120 bytes} [Mon Jul 25 12:03:02 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:03:02.484264 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42638257-4bc0-4988-9077-2f9dd97828a0 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:03:02.573184 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:03:02.581157 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42638257-4bc0-4988-9077-2f9dd97828a0 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:03:02.613924 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedec546-4e7c-4fe3-9261-6c8340f562b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:03:02.614584 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedec546-4e7c-4fe3-9261-6c8340f562b5 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:03:02.614871 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedec546-4e7c-4fe3-9261-6c8340f562b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:03:02.615150 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedec546-4e7c-4fe3-9261-6c8340f562b5 None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:03:02.615929 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dedec546-4e7c-4fe3-9261-6c8340f562b5 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:03:02.616238 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dedec546-4e7c-4fe3-9261-6c8340f562b5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:03:02.617604 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dedec546-4e7c-4fe3-9261-6c8340f562b5 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:03:02.625812 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 542/1088] 10.222.0.42 () {56 vars in 1115 bytes} [Mon Jul 25 12:03:02 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:03:02.631429 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351180ea-9f3a-4844-8f2f-a6ed85b7cd74 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:03:02.731365 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351180ea-9f3a-4844-8f2f-a6ed85b7cd74 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:03:02.765273 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdaeef3d-b8d3-4b75-ac64-eddffb1877bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:03:02.766098 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaeef3d-b8d3-4b75-ac64-eddffb1877bf None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:03:02.766385 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaeef3d-b8d3-4b75-ac64-eddffb1877bf None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:03:02.766667 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaeef3d-b8d3-4b75-ac64-eddffb1877bf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:03:02.778807 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdaeef3d-b8d3-4b75-ac64-eddffb1877bf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:03:02.780297 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdaeef3d-b8d3-4b75-ac64-eddffb1877bf None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:03:02.798744 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-cdaeef3d-b8d3-4b75-ac64-eddffb1877bf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:03:02.798744 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:03:02.798744 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:03:02.798744 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:03:02.798744 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:03:02.798744 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:03:02.798744 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:03:02.798744 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:03:02.798744 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:03:02.798744 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:03:02.798744 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:03:02.798744 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:03:02.798744 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 12:03:02.798744 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 12:03:02.798744 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 12:03:02.799803 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 12:03:02.799803 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:03:02.799803 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:03:02.799803 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:03:02.799803 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:03:02.799803 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:03:02.799803 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:03:02.799803 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:03:02.799803 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jul 25 12:03:02.799803 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:03:02.799803 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:03:02.799803 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:03:02.799803 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:03:02.799803 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:03:02.799803 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:03:02.799803 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:03:02.799803 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:03:02.800727 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 12:03:02.800727 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 12:03:02.800727 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:03:02.800727 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jul 25 12:03:02.800727 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 547/1089] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jul 25 12:03:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:03:02.805739 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedec546-4e7c-4fe3-9261-6c8340f562b5 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:03:02.902777 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedec546-4e7c-4fe3-9261-6c8340f562b5 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:03:02.936664 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c83310-98c9-4d5d-986d-45018f1801da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:03:02.937349 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c83310-98c9-4d5d-986d-45018f1801da None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:03:02.937640 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c83310-98c9-4d5d-986d-45018f1801da None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:03:02.937924 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c83310-98c9-4d5d-986d-45018f1801da None admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:03:02.938651 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c83310-98c9-4d5d-986d-45018f1801da None admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:03:02.938970 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c83310-98c9-4d5d-986d-45018f1801da None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:03:02.940371 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c83310-98c9-4d5d-986d-45018f1801da None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:03:02.951957 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 543/1090] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jul 25 12:03:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:03:02.957546 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdaeef3d-b8d3-4b75-ac64-eddffb1877bf None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:03:02.986458 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:03:03.057599 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdaeef3d-b8d3-4b75-ac64-eddffb1877bf None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:03:03.090562 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fc7f5c-9499-4e15-b55c-9c5ec58850b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:03:03.091353 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fc7f5c-9499-4e15-b55c-9c5ec58850b0 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:03:03.091681 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fc7f5c-9499-4e15-b55c-9c5ec58850b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:03:03.091971 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fc7f5c-9499-4e15-b55c-9c5ec58850b0 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:03:03.100096 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42fc7f5c-9499-4e15-b55c-9c5ec58850b0 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:03:03.101536 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42fc7f5c-9499-4e15-b55c-9c5ec58850b0 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:03:03.115376 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-42fc7f5c-9499-4e15-b55c-9c5ec58850b0 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 25 12:03:03.115376 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:03:03.115376 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:03:03.115376 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:03:03.115376 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:03:03.115376 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:03:03.115376 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:03:03.115376 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:03:03.115376 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:03:03.115376 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:03:03.115376 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:03:03.115376 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:03:03.115376 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 12:03:03.115376 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 12:03:03.115376 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 12:03:03.115376 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 12:03:03.116435 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:03:03.116435 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:03:03.116435 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 12:03:03.116435 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 12:03:03.116435 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 12:03:03.116435 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 12:03:03.116435 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:03:03.116435 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:03:03.116435 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:03:03.116435 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:03:03.116435 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:03:03.116435 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jul 25 12:03:03.116435 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:03:03.116435 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:03:03.116435 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:03:03.116435 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:03:03.116435 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:03:03.116435 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:03:03.117363 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:03:03.117363 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:03:03.117363 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 12:03:03.117363 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 12:03:03.117363 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 12:03:03.117363 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 12:03:03.117363 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 12:03:03.117363 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 12:03:03.117363 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 25 12:03:03.117363 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jul 25 12:03:03.117363 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 548/1091] 10.222.0.42 () {56 vars in 1093 bytes} [Mon Jul 25 12:03:02 2022] GET /identity/v3/users/nova => generated 82 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:03:03.122417 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c83310-98c9-4d5d-986d-45018f1801da None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:03:03.143699 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:03:03.223726 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c83310-98c9-4d5d-986d-45018f1801da None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:03:03.256472 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde570ae-c6ff-460e-9385-1aa294e426a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:03:03.257075 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde570ae-c6ff-460e-9385-1aa294e426a3 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:03:03.257368 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde570ae-c6ff-460e-9385-1aa294e426a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:03:03.257658 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde570ae-c6ff-460e-9385-1aa294e426a3 None admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:03:03.258498 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde570ae-c6ff-460e-9385-1aa294e426a3 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:03:03.258816 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde570ae-c6ff-460e-9385-1aa294e426a3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:03:03.260276 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde570ae-c6ff-460e-9385-1aa294e426a3 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:03:03.277491 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 544/1092] 10.222.0.42 () {56 vars in 1133 bytes} [Mon Jul 25 12:03:03 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:03:03.283860 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fc7f5c-9499-4e15-b55c-9c5ec58850b0 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:03:03.370950 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:03:03.384063 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fc7f5c-9499-4e15-b55c-9c5ec58850b0 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:03:03.417193 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3253fc4-0c40-4856-9739-c2a78190bbe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:03:03.417840 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3253fc4-0c40-4856-9739-c2a78190bbe9 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:03:03.418132 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3253fc4-0c40-4856-9739-c2a78190bbe9 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:03:03.418417 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3253fc4-0c40-4856-9739-c2a78190bbe9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:03:03.487398 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3253fc4-0c40-4856-9739-c2a78190bbe9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:03:03.488927 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3253fc4-0c40-4856-9739-c2a78190bbe9 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:03:03.573908 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:03:03.586789 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 549/1093] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Jul 25 12:03:03 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 25 12:03:03.592652 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde570ae-c6ff-460e-9385-1aa294e426a3 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:03:03.665463 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:03:03.669911 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cede1422-ab9b-4462-98cc-cbca8aae007e None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:03:03.670207 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cede1422-ab9b-4462-98cc-cbca8aae007e None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:03:03.670492 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cede1422-ab9b-4462-98cc-cbca8aae007e None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:03:03.672497 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 550/1094] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 25 12:03:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 12:03:03.677463 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199adb0d-b5b5-4e24-a9aa-a6362d11862c None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:03:03.677753 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199adb0d-b5b5-4e24-a9aa-a6362d11862c None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:03:03.678035 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199adb0d-b5b5-4e24-a9aa-a6362d11862c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:03:03.691881 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde570ae-c6ff-460e-9385-1aa294e426a3 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:03:03.695680 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-199adb0d-b5b5-4e24-a9aa-a6362d11862c None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:03:03.696951 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-199adb0d-b5b5-4e24-a9aa-a6362d11862c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:03:03.725178 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d8dd0d-260f-47b9-b214-785de1f0896a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:03:03.725963 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d8dd0d-260f-47b9-b214-785de1f0896a None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:03:03.726250 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d8dd0d-260f-47b9-b214-785de1f0896a None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:03:03.726532 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d8dd0d-260f-47b9-b214-785de1f0896a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:03:03.736952 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d8dd0d-260f-47b9-b214-785de1f0896a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:03:03.738398 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d8dd0d-260f-47b9-b214-785de1f0896a None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:03:03.756192 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-f0d8dd0d-260f-47b9-b214-785de1f0896a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:03:03.756192 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:03:03.756192 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:03:03.756192 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:03:03.756192 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:03:03.756192 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:03:03.756192 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:03:03.756192 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:03:03.756192 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:03:03.756192 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:03:03.756192 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:03:03.756192 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:03:03.756192 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 12:03:03.756192 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 12:03:03.756192 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 12:03:03.757116 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 12:03:03.757116 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:03:03.757116 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:03:03.757116 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:03:03.757116 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:03:03.757116 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:03:03.757116 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:03:03.757116 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:03:03.757116 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 25 12:03:03.757116 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:03:03.757116 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:03:03.757116 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:03:03.757116 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:03:03.757116 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:03:03.757116 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:03:03.757116 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:03:03.757116 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:03:03.758018 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 12:03:03.758018 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 12:03:03.758018 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:03:03.758018 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 25 12:03:03.758018 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 545/1095] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jul 25 12:03:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:03:03.763106 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d8dd0d-260f-47b9-b214-785de1f0896a None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:03:03.768487 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 551/1096] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 25 12:03:03 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 25 12:03:03.791720 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199adb0d-b5b5-4e24-a9aa-a6362d11862c None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:03:03.862831 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d8dd0d-260f-47b9-b214-785de1f0896a None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:03:03.887401 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:03:03.888004 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199adb0d-b5b5-4e24-a9aa-a6362d11862c None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:03:03.897372 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d4b9dd-bad0-484b-a67d-1fb8e0f47397 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:03:03.898037 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d4b9dd-bad0-484b-a67d-1fb8e0f47397 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:03:03.898328 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d4b9dd-bad0-484b-a67d-1fb8e0f47397 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:03:03.898616 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d4b9dd-bad0-484b-a67d-1fb8e0f47397 None admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:03:03.899345 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d4b9dd-bad0-484b-a67d-1fb8e0f47397 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:03:03.899688 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d4b9dd-bad0-484b-a67d-1fb8e0f47397 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:03:03.901062 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d4b9dd-bad0-484b-a67d-1fb8e0f47397 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:03:03.907440 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:03:03.913037 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 546/1097] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jul 25 12:03:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 12:03:03.918983 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d4b9dd-bad0-484b-a67d-1fb8e0f47397 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:03:03.927264 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ea1e1d-9172-41b6-9572-cf19e95a8f50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:03:03.928102 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ea1e1d-9172-41b6-9572-cf19e95a8f50 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:03:03.928388 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ea1e1d-9172-41b6-9572-cf19e95a8f50 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:03:03.928669 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ea1e1d-9172-41b6-9572-cf19e95a8f50 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:03:03.939176 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ea1e1d-9172-41b6-9572-cf19e95a8f50 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 12:03:03.946770 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-f9ea1e1d-9172-41b6-9572-cf19e95a8f50 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 12:03:03.946770 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:03:03.946770 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:03:03.946770 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:03:03.946770 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:03:03.946770 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:03:03.946770 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:03:03.946770 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:03:03.946770 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:03:03.946770 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:03:03.946770 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:03:03.946770 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:03:03.946770 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 12:03:03.946770 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 12:03:03.946770 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 12:03:03.946770 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 12:03:03.947738 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 12:03:03.947738 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 12:03:03.947738 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 12:03:03.947738 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 12:03:03.947738 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:03:03.947738 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:03:03.947738 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:03:03.947738 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:03:03.947738 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:03:03.947738 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:03:03.947738 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:03:03.947738 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jul 25 12:03:03.947738 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:03:03.947738 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:03:03.947738 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:03:03.947738 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:03:03.948579 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:03:03.948579 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:03:03.948579 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:03:03.948579 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:03:03.948579 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 12:03:03.948579 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 12:03:03.948579 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 12:03:03.948579 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 12:03:03.948579 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 12:03:03.948579 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 12:03:03.948579 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 12:03:03.948579 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jul 25 12:03:03.953525 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 552/1098] 10.222.0.42 () {56 vars in 1096 bytes} [Mon Jul 25 12:03:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 12:03:03.954283 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ea1e1d-9172-41b6-9572-cf19e95a8f50 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:03:04.016315 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d4b9dd-bad0-484b-a67d-1fb8e0f47397 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:03:04.049897 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d663590a-09ce-4fcc-9aac-43c66b02384e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:03:04.050711 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d663590a-09ce-4fcc-9aac-43c66b02384e None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:03:04.050999 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d663590a-09ce-4fcc-9aac-43c66b02384e None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:03:04.051278 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d663590a-09ce-4fcc-9aac-43c66b02384e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:03:04.051493 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ea1e1d-9172-41b6-9572-cf19e95a8f50 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:03:04.058372 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d663590a-09ce-4fcc-9aac-43c66b02384e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:03:04.059831 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d663590a-09ce-4fcc-9aac-43c66b02384e None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:03:04.073674 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-d663590a-09ce-4fcc-9aac-43c66b02384e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 12:03:04.073674 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:03:04.073674 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:03:04.073674 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:03:04.073674 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:03:04.073674 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:03:04.073674 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:03:04.073674 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:03:04.073674 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:03:04.073674 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:03:04.073674 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:03:04.073674 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:03:04.073674 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 12:03:04.073674 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 12:03:04.073674 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 12:03:04.074732 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 12:03:04.074732 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:03:04.074732 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:03:04.074732 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:03:04.074732 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:03:04.074732 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:03:04.074732 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:03:04.074732 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:03:04.074732 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 25 12:03:04.074732 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:03:04.074732 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:03:04.074732 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:03:04.074732 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:03:04.074732 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:03:04.074732 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:03:04.074732 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:03:04.074732 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:03:04.075871 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 12:03:04.075871 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 12:03:04.075871 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 12:03:04.075871 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 12:03:04.075871 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 12:03:04.075871 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 12:03:04.075871 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 12:03:04.075871 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 25 12:03:04.075871 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 547/1099] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Jul 25 12:03:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 12:03:04.081092 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d663590a-09ce-4fcc-9aac-43c66b02384e None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:03:04.086796 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d05f232-d5f6-4584-a734-45aaef7a4965 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:03:04.087458 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d05f232-d5f6-4584-a734-45aaef7a4965 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:03:04.087792 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d05f232-d5f6-4584-a734-45aaef7a4965 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:03:04.088073 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d05f232-d5f6-4584-a734-45aaef7a4965 None admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:03:04.088825 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d05f232-d5f6-4584-a734-45aaef7a4965 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:03:04.089142 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d05f232-d5f6-4584-a734-45aaef7a4965 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:03:04.090497 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d05f232-d5f6-4584-a734-45aaef7a4965 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:03:04.099193 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 553/1100] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Jul 25 12:03:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 12:03:04.104769 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d05f232-d5f6-4584-a734-45aaef7a4965 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:03:04.127182 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:03:04.147066 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:03:04.181132 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d663590a-09ce-4fcc-9aac-43c66b02384e None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:03:04.206349 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d05f232-d5f6-4584-a734-45aaef7a4965 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:03:04.215365 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a22905-2229-45ad-80b2-20fe028d986a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:03:04.216070 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a22905-2229-45ad-80b2-20fe028d986a None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:03:04.216369 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a22905-2229-45ad-80b2-20fe028d986a None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:03:04.216656 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a22905-2229-45ad-80b2-20fe028d986a None admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:03:04.217447 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a22905-2229-45ad-80b2-20fe028d986a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:03:04.217767 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a22905-2229-45ad-80b2-20fe028d986a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:03:04.219741 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a22905-2229-45ad-80b2-20fe028d986a None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:03:04.232393 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 548/1101] 10.222.0.42 () {56 vars in 1148 bytes} [Mon Jul 25 12:03:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 12:03:04.242491 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf258df-0fd0-45b5-bad1-4c99a32a4d23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:03:04.242491 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf258df-0fd0-45b5-bad1-4c99a32a4d23 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:03:04.242491 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf258df-0fd0-45b5-bad1-4c99a32a4d23 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:03:04.242491 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf258df-0fd0-45b5-bad1-4c99a32a4d23 None admin] PATH_INFO: `/v3/users/09e1099cc2be4c10a4d99bf2e2f1d458` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:03:04.242975 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a22905-2229-45ad-80b2-20fe028d986a None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:03:04.243618 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf258df-0fd0-45b5-bad1-4c99a32a4d23 None admin] RBAC: Authorizing `identity:get_user(user_id=09e1099cc2be4c10a4d99bf2e2f1d458)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:03:04.245046 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf258df-0fd0-45b5-bad1-4c99a32a4d23 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:03:04.246680 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 554/1102] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Jul 25 12:03:04 2022] GET /identity/v3/users/09e1099cc2be4c10a4d99bf2e2f1d458 => generated 285 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 12:03:04.252333 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf258df-0fd0-45b5-bad1-4c99a32a4d23 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:03:04.341436 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a22905-2229-45ad-80b2-20fe028d986a None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:03:04.348679 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:03:04.351122 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf258df-0fd0-45b5-bad1-4c99a32a4d23 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:03:04.360642 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:03:04.376154 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7488b737-1011-4402-ac29-af1498a8a763 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:03:04.376809 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7488b737-1011-4402-ac29-af1498a8a763 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:03:04.377103 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7488b737-1011-4402-ac29-af1498a8a763 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:03:04.377450 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7488b737-1011-4402-ac29-af1498a8a763 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:03:04.378210 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7488b737-1011-4402-ac29-af1498a8a763 None admin] RBAC: Adding query filter params (role.id=3297f946c8f04c9f8a159c7e36bf41a5, scope.project.id=b9f62b8f37304b8a9a4bb54205347577, user.id=f67142a1193d47c7bd33d7040268233c) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:03:04.378528 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7488b737-1011-4402-ac29-af1498a8a763 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:03:04.380011 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7488b737-1011-4402-ac29-af1498a8a763 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:03:04.385692 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5201c272-e3ef-42bb-8b6b-e54f15e95fcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:03:04.385820 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 549/1103] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Jul 25 12:03:04 2022] GET /identity/v3/role_assignments?user.id=f67142a1193d47c7bd33d7040268233c&scope.project.id=b9f62b8f37304b8a9a4bb54205347577&role.id=3297f946c8f04c9f8a159c7e36bf41a5 => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 12:03:04.386341 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5201c272-e3ef-42bb-8b6b-e54f15e95fcb None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:03:04.386627 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5201c272-e3ef-42bb-8b6b-e54f15e95fcb None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:03:04.386910 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5201c272-e3ef-42bb-8b6b-e54f15e95fcb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:03:04.459253 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5201c272-e3ef-42bb-8b6b-e54f15e95fcb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:03:04.460761 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5201c272-e3ef-42bb-8b6b-e54f15e95fcb None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:03:04.565738 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 555/1104] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Jul 25 12:03:04 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 25 12:03:04.571733 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7488b737-1011-4402-ac29-af1498a8a763 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:03:04.675431 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7488b737-1011-4402-ac29-af1498a8a763 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:03:04.708817 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2dfa377-bfaf-4c3c-b3dd-efb64552cad3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:03:04.709672 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dfa377-bfaf-4c3c-b3dd-efb64552cad3 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:03:04.709999 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dfa377-bfaf-4c3c-b3dd-efb64552cad3 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:03:04.710224 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dfa377-bfaf-4c3c-b3dd-efb64552cad3 None admin] PATH_INFO: `/v3/projects/bff2d22cbce449d9ac721dd3cfd8b074` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:03:04.711386 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2dfa377-bfaf-4c3c-b3dd-efb64552cad3 None admin] RBAC: Authorizing `identity:get_project(project_id=bff2d22cbce449d9ac721dd3cfd8b074)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:03:04.712861 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2dfa377-bfaf-4c3c-b3dd-efb64552cad3 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:03:04.714892 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 550/1105] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Jul 25 12:03:04 2022] GET /identity/v3/projects/bff2d22cbce449d9ac721dd3cfd8b074 => generated 311 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:03:04.720737 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5201c272-e3ef-42bb-8b6b-e54f15e95fcb None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:03:04.730986 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:03:04.827929 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5201c272-e3ef-42bb-8b6b-e54f15e95fcb None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:03:04.863098 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcfc97e-07b6-4a0d-b330-d5cec15e933e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:03:04.863794 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcfc97e-07b6-4a0d-b330-d5cec15e933e None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:03:04.864084 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcfc97e-07b6-4a0d-b330-d5cec15e933e None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:03:04.864367 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcfc97e-07b6-4a0d-b330-d5cec15e933e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:03:04.865191 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fcfc97e-07b6-4a0d-b330-d5cec15e933e None admin] RBAC: Adding query filter params (role.id=a7d20885b6d94928a34b3cc5b3a00006, scope.project.id=bff2d22cbce449d9ac721dd3cfd8b074, user.id=09e1099cc2be4c10a4d99bf2e2f1d458) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:03:04.865509 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fcfc97e-07b6-4a0d-b330-d5cec15e933e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:03:04.866930 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fcfc97e-07b6-4a0d-b330-d5cec15e933e None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:03:04.873455 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 556/1106] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Jul 25 12:03:04 2022] GET /identity/v3/role_assignments?user.id=09e1099cc2be4c10a4d99bf2e2f1d458&scope.project.id=bff2d22cbce449d9ac721dd3cfd8b074&role.id=a7d20885b6d94928a34b3cc5b3a00006 => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:03:06.881703 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a353fd79-2aee-4e61-b508-5fb7a20ea258 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:03:06.882222 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a353fd79-2aee-4e61-b508-5fb7a20ea258 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:03:06.882317 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a353fd79-2aee-4e61-b508-5fb7a20ea258 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:03:06.883672 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 551/1107] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 25 12:03:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:03:06.888988 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f31ed76-b0ea-4c0b-94ff-b7e057d10bb6 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:03:06.889385 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f31ed76-b0ea-4c0b-94ff-b7e057d10bb6 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:03:06.889549 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f31ed76-b0ea-4c0b-94ff-b7e057d10bb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:03:06.907203 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-7f31ed76-b0ea-4c0b-94ff-b7e057d10bb6 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:03:06.908429 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-7f31ed76-b0ea-4c0b-94ff-b7e057d10bb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:03:06.973055 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 557/1108] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 25 12:03:06 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 12:03:06.997535 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a353fd79-2aee-4e61-b508-5fb7a20ea258 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:03:07.038518 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:03:07.098491 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a353fd79-2aee-4e61-b508-5fb7a20ea258 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:03:07.131678 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9505696-3785-418f-8d77-876cceccc0a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:03:07.132456 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9505696-3785-418f-8d77-876cceccc0a4 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:03:07.132743 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9505696-3785-418f-8d77-876cceccc0a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:03:07.133025 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9505696-3785-418f-8d77-876cceccc0a4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:03:07.134195 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9505696-3785-418f-8d77-876cceccc0a4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:03:07.135659 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9505696-3785-418f-8d77-876cceccc0a4 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:03:07.137252 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 552/1109] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jul 25 12:03:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:03:07.142829 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f31ed76-b0ea-4c0b-94ff-b7e057d10bb6 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:03:07.145041 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:03:07.244341 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f31ed76-b0ea-4c0b-94ff-b7e057d10bb6 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:03:07.276729 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb62988-dd7a-4ed5-a9af-319afd998d38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:03:07.277395 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb62988-dd7a-4ed5-a9af-319afd998d38 None admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:03:07.277691 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb62988-dd7a-4ed5-a9af-319afd998d38 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:03:07.277973 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb62988-dd7a-4ed5-a9af-319afd998d38 None admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:03:07.278695 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb62988-dd7a-4ed5-a9af-319afd998d38 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:03:07.280166 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb62988-dd7a-4ed5-a9af-319afd998d38 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:03:07.302563 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 558/1110] 10.222.0.42 () {60 vars in 1132 bytes} [Mon Jul 25 12:03:07 2022] POST /identity/v3/users => generated 286 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 12:03:09.314712 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937f42dc-dc3e-48d0-ac8d-8c70a9db94f4 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:03:09.315240 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937f42dc-dc3e-48d0-ac8d-8c70a9db94f4 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:03:09.315346 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937f42dc-dc3e-48d0-ac8d-8c70a9db94f4 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:03:09.316671 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 553/1111] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 25 12:03:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:03:09.321975 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f64a76-a272-4e70-bd9e-fcb75cff6645 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:03:09.322390 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f64a76-a272-4e70-bd9e-fcb75cff6645 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:03:09.322542 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f64a76-a272-4e70-bd9e-fcb75cff6645 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:03:09.341420 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-48f64a76-a272-4e70-bd9e-fcb75cff6645 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:03:09.342608 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-48f64a76-a272-4e70-bd9e-fcb75cff6645 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:03:09.395351 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:03:09.409435 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 559/1112] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 25 12:03:09 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 12:03:09.432649 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937f42dc-dc3e-48d0-ac8d-8c70a9db94f4 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:03:09.532268 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937f42dc-dc3e-48d0-ac8d-8c70a9db94f4 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:03:09.539058 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:03:09.566400 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3eb7f84-9f74-4cce-a46d-fd0067e28dd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:03:09.567189 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3eb7f84-9f74-4cce-a46d-fd0067e28dd5 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:03:09.567479 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3eb7f84-9f74-4cce-a46d-fd0067e28dd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:03:09.567801 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3eb7f84-9f74-4cce-a46d-fd0067e28dd5 None admin] PATH_INFO: `/v3/roles/dea3ef72822043f3baf3bcc1be7c318f` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:03:09.569277 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3eb7f84-9f74-4cce-a46d-fd0067e28dd5 None admin] RBAC: Authorizing `identity:get_role(role_id=dea3ef72822043f3baf3bcc1be7c318f)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:03:09.570641 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3eb7f84-9f74-4cce-a46d-fd0067e28dd5 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:03:09.572027 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 554/1113] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Jul 25 12:03:09 2022] GET /identity/v3/roles/dea3ef72822043f3baf3bcc1be7c318f => generated 223 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:03:09.578466 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f64a76-a272-4e70-bd9e-fcb75cff6645 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:03:09.673151 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f64a76-a272-4e70-bd9e-fcb75cff6645 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:03:09.706285 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c513955-f03b-4689-9567-f46e3d3bcb49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:03:09.707133 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c513955-f03b-4689-9567-f46e3d3bcb49 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:03:09.707420 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c513955-f03b-4689-9567-f46e3d3bcb49 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:03:09.707733 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c513955-f03b-4689-9567-f46e3d3bcb49 None admin] PATH_INFO: `/v3/users/4e0a6a7262d244b6b6bd55c3acf18a6e` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:03:09.720316 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c513955-f03b-4689-9567-f46e3d3bcb49 None admin] RBAC: Authorizing `identity:get_user(user_id=4e0a6a7262d244b6b6bd55c3acf18a6e)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:03:09.721755 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c513955-f03b-4689-9567-f46e3d3bcb49 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:03:09.723352 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 560/1114] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Jul 25 12:03:09 2022] GET /identity/v3/users/4e0a6a7262d244b6b6bd55c3acf18a6e => generated 286 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 12:03:09.728920 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3eb7f84-9f74-4cce-a46d-fd0067e28dd5 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:03:09.831297 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3eb7f84-9f74-4cce-a46d-fd0067e28dd5 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:03:09.863942 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413dff16-a6ab-440f-937f-242cafcb9443 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:03:09.864596 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413dff16-a6ab-440f-937f-242cafcb9443 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:03:09.864890 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413dff16-a6ab-440f-937f-242cafcb9443 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:03:09.865178 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413dff16-a6ab-440f-937f-242cafcb9443 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:03:09.905725 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:03:09.931179 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413dff16-a6ab-440f-937f-242cafcb9443 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:03:09.932669 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413dff16-a6ab-440f-937f-242cafcb9443 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:03:10.026900 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 555/1115] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Jul 25 12:03:09 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 25 12:03:10.032973 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c513955-f03b-4689-9567-f46e3d3bcb49 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:03:10.079065 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:03:10.131522 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c513955-f03b-4689-9567-f46e3d3bcb49 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:03:10.164201 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831d73a7-9908-451d-a03f-f3219c439309 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:03:10.165305 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831d73a7-9908-451d-a03f-f3219c439309 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:03:10.165593 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831d73a7-9908-451d-a03f-f3219c439309 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:03:10.165875 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831d73a7-9908-451d-a03f-f3219c439309 None admin] PATH_INFO: `/v3/projects/bff2d22cbce449d9ac721dd3cfd8b074` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:03:10.167048 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831d73a7-9908-451d-a03f-f3219c439309 None admin] RBAC: Authorizing `identity:get_project(project_id=bff2d22cbce449d9ac721dd3cfd8b074)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:03:10.168509 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831d73a7-9908-451d-a03f-f3219c439309 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:03:10.170178 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 561/1116] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Jul 25 12:03:10 2022] GET /identity/v3/projects/bff2d22cbce449d9ac721dd3cfd8b074 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:03:10.176945 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413dff16-a6ab-440f-937f-242cafcb9443 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:03:10.251340 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:03:10.275795 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413dff16-a6ab-440f-937f-242cafcb9443 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:03:10.312289 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8ebfa8-b4e3-46cd-9dd4-d07bf957149f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:03:10.312933 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8ebfa8-b4e3-46cd-9dd4-d07bf957149f None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:03:10.313215 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8ebfa8-b4e3-46cd-9dd4-d07bf957149f None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:03:10.313492 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8ebfa8-b4e3-46cd-9dd4-d07bf957149f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:03:10.314317 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8ebfa8-b4e3-46cd-9dd4-d07bf957149f None admin] RBAC: Adding query filter params (role.id=dea3ef72822043f3baf3bcc1be7c318f, scope.project.id=bff2d22cbce449d9ac721dd3cfd8b074, user.id=4e0a6a7262d244b6b6bd55c3acf18a6e) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:03:10.314628 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8ebfa8-b4e3-46cd-9dd4-d07bf957149f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:03:10.316096 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8ebfa8-b4e3-46cd-9dd4-d07bf957149f None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:03:10.321316 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 556/1117] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Jul 25 12:03:10 2022] GET /identity/v3/role_assignments?user.id=4e0a6a7262d244b6b6bd55c3acf18a6e&scope.project.id=bff2d22cbce449d9ac721dd3cfd8b074&role.id=dea3ef72822043f3baf3bcc1be7c318f => generated 259 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:03:12.303401 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1392fd-db16-4679-9d2c-2c246dd8f492 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:03:12.303989 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1392fd-db16-4679-9d2c-2c246dd8f492 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:03:12.303989 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1392fd-db16-4679-9d2c-2c246dd8f492 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:03:12.305359 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 562/1118] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 25 12:03:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:03:12.310544 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f2b737-db79-4aba-9780-6a0ff82f39e8 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:03:12.310907 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f2b737-db79-4aba-9780-6a0ff82f39e8 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:03:12.311117 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f2b737-db79-4aba-9780-6a0ff82f39e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:03:12.329924 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-f1f2b737-db79-4aba-9780-6a0ff82f39e8 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:03:12.331110 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-f1f2b737-db79-4aba-9780-6a0ff82f39e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:03:12.396011 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 557/1119] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 25 12:03:12 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 12:03:12.417931 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1392fd-db16-4679-9d2c-2c246dd8f492 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:03:12.523374 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1392fd-db16-4679-9d2c-2c246dd8f492 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:03:12.537162 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:03:12.557481 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329c5722-e1e0-4acb-b3a4-5edba96ff986 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:03:12.558275 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329c5722-e1e0-4acb-b3a4-5edba96ff986 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:03:12.558568 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329c5722-e1e0-4acb-b3a4-5edba96ff986 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:03:12.558852 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329c5722-e1e0-4acb-b3a4-5edba96ff986 None admin] PATH_INFO: `/v3/roles/dea3ef72822043f3baf3bcc1be7c318f` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:03:12.560377 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329c5722-e1e0-4acb-b3a4-5edba96ff986 None admin] RBAC: Authorizing `identity:get_role(role_id=dea3ef72822043f3baf3bcc1be7c318f)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:03:12.562267 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329c5722-e1e0-4acb-b3a4-5edba96ff986 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:03:12.563654 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 563/1120] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Jul 25 12:03:12 2022] GET /identity/v3/roles/dea3ef72822043f3baf3bcc1be7c318f => generated 223 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:03:12.569505 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f2b737-db79-4aba-9780-6a0ff82f39e8 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:03:12.632378 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:03:12.664755 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f2b737-db79-4aba-9780-6a0ff82f39e8 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:03:12.697184 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eea3aa0-567a-477b-be07-212ec98b339d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:03:12.698031 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eea3aa0-567a-477b-be07-212ec98b339d None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:03:12.698455 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eea3aa0-567a-477b-be07-212ec98b339d None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:03:12.698737 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eea3aa0-567a-477b-be07-212ec98b339d None admin] PATH_INFO: `/v3/users/4e0a6a7262d244b6b6bd55c3acf18a6e` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:03:12.699932 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eea3aa0-567a-477b-be07-212ec98b339d None admin] RBAC: Authorizing `identity:get_user(user_id=4e0a6a7262d244b6b6bd55c3acf18a6e)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:03:12.701377 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eea3aa0-567a-477b-be07-212ec98b339d None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:03:12.703583 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 558/1121] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Jul 25 12:03:12 2022] GET /identity/v3/users/4e0a6a7262d244b6b6bd55c3acf18a6e => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:03:12.709108 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329c5722-e1e0-4acb-b3a4-5edba96ff986 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:03:12.813238 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329c5722-e1e0-4acb-b3a4-5edba96ff986 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:03:12.845840 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353a14c6-7ffd-48b7-91e4-6358d767e251 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:03:12.846682 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353a14c6-7ffd-48b7-91e4-6358d767e251 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:03:12.846963 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353a14c6-7ffd-48b7-91e4-6358d767e251 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:03:12.847234 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353a14c6-7ffd-48b7-91e4-6358d767e251 None admin] PATH_INFO: `/v3/projects/bff2d22cbce449d9ac721dd3cfd8b074` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:03:12.848443 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353a14c6-7ffd-48b7-91e4-6358d767e251 None admin] RBAC: Authorizing `identity:get_project(project_id=bff2d22cbce449d9ac721dd3cfd8b074)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:03:12.849865 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353a14c6-7ffd-48b7-91e4-6358d767e251 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:03:12.851538 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 564/1122] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Jul 25 12:03:12 2022] GET /identity/v3/projects/bff2d22cbce449d9ac721dd3cfd8b074 => generated 311 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:03:12.857497 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eea3aa0-567a-477b-be07-212ec98b339d None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:03:12.955069 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eea3aa0-567a-477b-be07-212ec98b339d None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:03:12.987870 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ade6f06-7018-4c30-bbff-45e05f559aae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:03:12.988748 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ade6f06-7018-4c30-bbff-45e05f559aae None admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:03:12.989035 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ade6f06-7018-4c30-bbff-45e05f559aae None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:03:12.989317 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ade6f06-7018-4c30-bbff-45e05f559aae None admin] PATH_INFO: `/v3/projects/bff2d22cbce449d9ac721dd3cfd8b074/users/4e0a6a7262d244b6b6bd55c3acf18a6e/roles/dea3ef72822043f3baf3bcc1be7c318f` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:03:12.991511 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ade6f06-7018-4c30-bbff-45e05f559aae None admin] RBAC: Authorizing `identity:create_grant(project_id=bff2d22cbce449d9ac721dd3cfd8b074, user_id=4e0a6a7262d244b6b6bd55c3acf18a6e, role_id=dea3ef72822043f3baf3bcc1be7c318f)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:03:12.992960 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ade6f06-7018-4c30-bbff-45e05f559aae None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:03:12.995951 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:03:13.007936 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 559/1123] 10.222.0.42 () {58 vars in 1439 bytes} [Mon Jul 25 12:03:12 2022] PUT /identity/v3/projects/bff2d22cbce449d9ac721dd3cfd8b074/users/4e0a6a7262d244b6b6bd55c3acf18a6e/roles/dea3ef72822043f3baf3bcc1be7c318f => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:03:14.992975 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57568679-a8d9-41b2-92e3-6850c2895cba None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:03:14.993560 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57568679-a8d9-41b2-92e3-6850c2895cba None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:03:14.993560 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57568679-a8d9-41b2-92e3-6850c2895cba None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:03:14.994891 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 565/1124] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 25 12:03:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:03:15.000201 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9033dad5-108b-404e-a36a-677995901a93 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:03:15.000610 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9033dad5-108b-404e-a36a-677995901a93 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:03:15.000768 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9033dad5-108b-404e-a36a-677995901a93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:03:15.018945 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-9033dad5-108b-404e-a36a-677995901a93 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:03:15.020166 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-9033dad5-108b-404e-a36a-677995901a93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:03:15.086655 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 560/1125] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 25 12:03:14 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 12:03:15.110034 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57568679-a8d9-41b2-92e3-6850c2895cba None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:03:15.160177 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:03:15.209634 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57568679-a8d9-41b2-92e3-6850c2895cba None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:03:15.243043 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f6e734-f928-4cb6-afe3-c03b661c10f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:03:15.243903 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f6e734-f928-4cb6-afe3-c03b661c10f7 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:03:15.244188 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f6e734-f928-4cb6-afe3-c03b661c10f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:03:15.244465 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f6e734-f928-4cb6-afe3-c03b661c10f7 None admin] PATH_INFO: `/v3/roles/dea3ef72822043f3baf3bcc1be7c318f` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:03:15.245911 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f6e734-f928-4cb6-afe3-c03b661c10f7 None admin] RBAC: Authorizing `identity:get_role(role_id=dea3ef72822043f3baf3bcc1be7c318f)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:03:15.247271 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f6e734-f928-4cb6-afe3-c03b661c10f7 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:03:15.248646 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 566/1126] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Jul 25 12:03:15 2022] GET /identity/v3/roles/dea3ef72822043f3baf3bcc1be7c318f => generated 223 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:03:15.254685 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9033dad5-108b-404e-a36a-677995901a93 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:03:15.350771 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9033dad5-108b-404e-a36a-677995901a93 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:03:15.375446 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:03:15.384347 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef9a76c-2457-4ed6-ae6b-185472be8631 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:03:15.385207 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef9a76c-2457-4ed6-ae6b-185472be8631 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:03:15.385496 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef9a76c-2457-4ed6-ae6b-185472be8631 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:03:15.385776 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef9a76c-2457-4ed6-ae6b-185472be8631 None admin] PATH_INFO: `/v3/users/4e0a6a7262d244b6b6bd55c3acf18a6e` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:03:15.386970 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef9a76c-2457-4ed6-ae6b-185472be8631 None admin] RBAC: Authorizing `identity:get_user(user_id=4e0a6a7262d244b6b6bd55c3acf18a6e)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:03:15.388440 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef9a76c-2457-4ed6-ae6b-185472be8631 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:03:15.390627 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 561/1127] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Jul 25 12:03:15 2022] GET /identity/v3/users/4e0a6a7262d244b6b6bd55c3acf18a6e => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:03:15.396209 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f6e734-f928-4cb6-afe3-c03b661c10f7 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:03:15.491443 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f6e734-f928-4cb6-afe3-c03b661c10f7 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:03:15.508529 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:03:15.525885 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eacf42a-1906-498a-b13a-0cbe78dd6d3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:03:15.526515 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eacf42a-1906-498a-b13a-0cbe78dd6d3e None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:03:15.526800 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eacf42a-1906-498a-b13a-0cbe78dd6d3e None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:03:15.527079 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eacf42a-1906-498a-b13a-0cbe78dd6d3e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:03:15.591285 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eacf42a-1906-498a-b13a-0cbe78dd6d3e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:03:15.592768 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eacf42a-1906-498a-b13a-0cbe78dd6d3e None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:03:15.686646 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 567/1128] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Jul 25 12:03:15 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 25 12:03:15.692490 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef9a76c-2457-4ed6-ae6b-185472be8631 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:03:15.788478 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef9a76c-2457-4ed6-ae6b-185472be8631 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:03:15.822202 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8786d680-2d28-4426-968b-81db3703cd80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:03:15.823058 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8786d680-2d28-4426-968b-81db3703cd80 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:03:15.823349 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8786d680-2d28-4426-968b-81db3703cd80 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:03:15.823657 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8786d680-2d28-4426-968b-81db3703cd80 None admin] PATH_INFO: `/v3/projects/bff2d22cbce449d9ac721dd3cfd8b074` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:03:15.824839 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8786d680-2d28-4426-968b-81db3703cd80 None admin] RBAC: Authorizing `identity:get_project(project_id=bff2d22cbce449d9ac721dd3cfd8b074)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:03:15.826272 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8786d680-2d28-4426-968b-81db3703cd80 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:03:15.828037 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 562/1129] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Jul 25 12:03:15 2022] GET /identity/v3/projects/bff2d22cbce449d9ac721dd3cfd8b074 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:03:15.834180 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eacf42a-1906-498a-b13a-0cbe78dd6d3e None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:03:15.852038 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:03:15.932430 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eacf42a-1906-498a-b13a-0cbe78dd6d3e None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:03:15.965574 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1467bbb-63b5-45e2-800e-e4de271585db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:03:15.966167 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1467bbb-63b5-45e2-800e-e4de271585db None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:03:15.966537 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1467bbb-63b5-45e2-800e-e4de271585db None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:03:15.966824 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1467bbb-63b5-45e2-800e-e4de271585db None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:03:15.967708 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1467bbb-63b5-45e2-800e-e4de271585db None admin] RBAC: Adding query filter params (role.id=dea3ef72822043f3baf3bcc1be7c318f, scope.project.id=bff2d22cbce449d9ac721dd3cfd8b074, user.id=4e0a6a7262d244b6b6bd55c3acf18a6e) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:03:15.968028 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1467bbb-63b5-45e2-800e-e4de271585db None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:03:15.969450 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1467bbb-63b5-45e2-800e-e4de271585db None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:03:15.975085 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 568/1130] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Jul 25 12:03:15 2022] GET /identity/v3/role_assignments?user.id=4e0a6a7262d244b6b6bd55c3acf18a6e&scope.project.id=bff2d22cbce449d9ac721dd3cfd8b074&role.id=dea3ef72822043f3baf3bcc1be7c318f => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:03:17.966055 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad900a4d-5ad8-4c46-916f-022f71678ffb None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:03:17.966590 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad900a4d-5ad8-4c46-916f-022f71678ffb None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:03:17.966590 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad900a4d-5ad8-4c46-916f-022f71678ffb None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:03:17.968050 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 563/1131] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 25 12:03:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:03:17.973351 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8e5b45-481b-451b-afb9-329a320cd208 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:03:17.973775 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8e5b45-481b-451b-afb9-329a320cd208 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:03:17.973913 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8e5b45-481b-451b-afb9-329a320cd208 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:03:17.992849 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-ea8e5b45-481b-451b-afb9-329a320cd208 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:03:17.994035 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-ea8e5b45-481b-451b-afb9-329a320cd208 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:03:18.061758 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 569/1132] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 25 12:03:17 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 12:03:18.084884 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad900a4d-5ad8-4c46-916f-022f71678ffb None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:03:18.145908 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:03:18.184681 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad900a4d-5ad8-4c46-916f-022f71678ffb None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:03:18.217528 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38532673-9a9e-4fbf-aac3-d2ab0fc1e0fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:03:18.218323 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38532673-9a9e-4fbf-aac3-d2ab0fc1e0fb None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:03:18.218611 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38532673-9a9e-4fbf-aac3-d2ab0fc1e0fb None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:03:18.218894 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38532673-9a9e-4fbf-aac3-d2ab0fc1e0fb None admin] PATH_INFO: `/v3/domains/default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:03:18.220106 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38532673-9a9e-4fbf-aac3-d2ab0fc1e0fb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:03:18.221563 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38532673-9a9e-4fbf-aac3-d2ab0fc1e0fb None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:03:18.223267 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 564/1133] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jul 25 12:03:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:03:18.229311 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8e5b45-481b-451b-afb9-329a320cd208 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:03:18.244647 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:03:18.336023 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8e5b45-481b-451b-afb9-329a320cd208 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:03:18.374371 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5e3524-e028-49ec-8dcb-e5461e2ec139 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:03:18.375025 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5e3524-e028-49ec-8dcb-e5461e2ec139 None admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:03:18.375314 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5e3524-e028-49ec-8dcb-e5461e2ec139 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:03:18.375621 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5e3524-e028-49ec-8dcb-e5461e2ec139 None admin] PATH_INFO: `/v3/projects` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:03:18.376341 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5e3524-e028-49ec-8dcb-e5461e2ec139 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:03:18.377767 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5e3524-e028-49ec-8dcb-e5461e2ec139 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:03:18.405975 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 570/1134] 10.222.0.42 () {60 vars in 1141 bytes} [Mon Jul 25 12:03:18 2022] POST /identity/v3/projects => generated 311 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:03:20.400827 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29216fe-2e36-47da-9502-840b69c7e014 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:03:20.401404 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29216fe-2e36-47da-9502-840b69c7e014 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:03:20.401404 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29216fe-2e36-47da-9502-840b69c7e014 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:03:20.402767 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 565/1135] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 25 12:03:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:03:20.408450 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc90849-15ca-4588-8303-32357ad1c2d0 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:03:20.408865 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc90849-15ca-4588-8303-32357ad1c2d0 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:03:20.409024 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc90849-15ca-4588-8303-32357ad1c2d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:03:20.427728 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-8dc90849-15ca-4588-8303-32357ad1c2d0 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:03:20.428917 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-8dc90849-15ca-4588-8303-32357ad1c2d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:03:20.485303 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:03:20.494559 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 571/1136] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 25 12:03:20 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 12:03:20.519491 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29216fe-2e36-47da-9502-840b69c7e014 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:03:20.617858 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29216fe-2e36-47da-9502-840b69c7e014 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:03:20.645920 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:03:20.651971 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310372c2-57a8-4d7e-a28b-6962c4f9213f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:03:20.652701 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310372c2-57a8-4d7e-a28b-6962c4f9213f None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:03:20.652988 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310372c2-57a8-4d7e-a28b-6962c4f9213f None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:03:20.653267 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310372c2-57a8-4d7e-a28b-6962c4f9213f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:03:20.654455 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310372c2-57a8-4d7e-a28b-6962c4f9213f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:03:20.655917 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310372c2-57a8-4d7e-a28b-6962c4f9213f None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:03:20.657509 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 566/1137] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jul 25 12:03:20 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:03:20.663073 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc90849-15ca-4588-8303-32357ad1c2d0 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:03:20.758565 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc90849-15ca-4588-8303-32357ad1c2d0 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:03:20.791876 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af2758c-1a5b-4095-8e2a-03fdf504cea3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:03:20.792543 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af2758c-1a5b-4095-8e2a-03fdf504cea3 None admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:03:20.792829 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af2758c-1a5b-4095-8e2a-03fdf504cea3 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:03:20.793109 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af2758c-1a5b-4095-8e2a-03fdf504cea3 None admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:03:20.793830 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af2758c-1a5b-4095-8e2a-03fdf504cea3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:03:20.795285 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af2758c-1a5b-4095-8e2a-03fdf504cea3 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:03:20.817065 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 572/1138] 10.222.0.42 () {60 vars in 1132 bytes} [Mon Jul 25 12:03:20 2022] POST /identity/v3/users => generated 286 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:03:22.863409 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3b43f5-c480-4001-a860-b7e238c4635a None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:03:22.864820 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3b43f5-c480-4001-a860-b7e238c4635a None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:03:22.864820 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3b43f5-c480-4001-a860-b7e238c4635a None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:03:22.865381 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 567/1139] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 25 12:03:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:03:22.870499 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d24b5d-72c4-41f7-9c33-a28b5261fde0 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:03:22.870897 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d24b5d-72c4-41f7-9c33-a28b5261fde0 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:03:22.871068 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d24b5d-72c4-41f7-9c33-a28b5261fde0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:03:22.890401 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-29d24b5d-72c4-41f7-9c33-a28b5261fde0 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:03:22.891626 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-29d24b5d-72c4-41f7-9c33-a28b5261fde0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:03:22.936472 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:03:22.958928 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 573/1140] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 25 12:03:22 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 12:03:22.981751 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3b43f5-c480-4001-a860-b7e238c4635a None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:03:23.080262 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3b43f5-c480-4001-a860-b7e238c4635a None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:03:23.112406 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df268185-29dc-490a-aaf7-8b1b1a4eeb8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:03:23.113180 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df268185-29dc-490a-aaf7-8b1b1a4eeb8b None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:03:23.113468 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df268185-29dc-490a-aaf7-8b1b1a4eeb8b None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:03:23.113753 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df268185-29dc-490a-aaf7-8b1b1a4eeb8b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:03:23.124157 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df268185-29dc-490a-aaf7-8b1b1a4eeb8b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 12:03:23.131660 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-df268185-29dc-490a-aaf7-8b1b1a4eeb8b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 12:03:23.131660 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:03:23.131660 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:03:23.131660 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:03:23.131660 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:03:23.131660 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:03:23.131660 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:03:23.131660 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:03:23.131660 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:03:23.131660 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:03:23.131660 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:03:23.131660 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:03:23.131660 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 12:03:23.131660 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 12:03:23.131660 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 12:03:23.131660 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 12:03:23.132820 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 12:03:23.132820 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 12:03:23.132820 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 12:03:23.132820 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 12:03:23.132820 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:03:23.132820 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:03:23.132820 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:03:23.132820 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:03:23.132820 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:03:23.132820 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:03:23.132820 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:03:23.132820 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 25 12:03:23.132820 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:03:23.132820 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:03:23.132820 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:03:23.132820 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:03:23.133905 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:03:23.133905 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:03:23.133905 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:03:23.133905 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:03:23.133905 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 12:03:23.133905 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 12:03:23.133905 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 12:03:23.133905 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 12:03:23.133905 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 12:03:23.133905 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 12:03:23.133905 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 12:03:23.133905 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 25 12:03:23.133905 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 568/1141] 10.222.0.42 () {56 vars in 1096 bytes} [Mon Jul 25 12:03:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:03:23.139250 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d24b5d-72c4-41f7-9c33-a28b5261fde0 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:03:23.233273 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d24b5d-72c4-41f7-9c33-a28b5261fde0 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:03:23.266559 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f9cbed-c62e-40e9-92fc-c8d8b3a40205 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:03:23.267209 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f9cbed-c62e-40e9-92fc-c8d8b3a40205 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:03:23.267495 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f9cbed-c62e-40e9-92fc-c8d8b3a40205 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:03:23.267801 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f9cbed-c62e-40e9-92fc-c8d8b3a40205 None admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:03:23.268536 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f9cbed-c62e-40e9-92fc-c8d8b3a40205 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:03:23.268844 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f9cbed-c62e-40e9-92fc-c8d8b3a40205 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:03:23.270198 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f9cbed-c62e-40e9-92fc-c8d8b3a40205 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:03:23.277960 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 574/1142] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Jul 25 12:03:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 12:03:23.283628 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df268185-29dc-490a-aaf7-8b1b1a4eeb8b None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:03:23.339942 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:03:23.382423 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df268185-29dc-490a-aaf7-8b1b1a4eeb8b None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:03:23.414057 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65387cdc-1c15-4a3e-9dd2-7c21598ac0c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:03:23.415169 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65387cdc-1c15-4a3e-9dd2-7c21598ac0c0 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:03:23.415454 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65387cdc-1c15-4a3e-9dd2-7c21598ac0c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:03:23.415759 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65387cdc-1c15-4a3e-9dd2-7c21598ac0c0 None admin] PATH_INFO: `/v3/users/831acca9e7ef488b9b3a50f50b045774` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:03:23.428995 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65387cdc-1c15-4a3e-9dd2-7c21598ac0c0 None admin] RBAC: Authorizing `identity:get_user(user_id=831acca9e7ef488b9b3a50f50b045774)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:03:23.430429 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65387cdc-1c15-4a3e-9dd2-7c21598ac0c0 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:03:23.432077 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 569/1143] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Jul 25 12:03:23 2022] GET /identity/v3/users/831acca9e7ef488b9b3a50f50b045774 => generated 286 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:03:23.438258 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f9cbed-c62e-40e9-92fc-c8d8b3a40205 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:03:23.479405 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:03:23.536223 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f9cbed-c62e-40e9-92fc-c8d8b3a40205 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:03:23.569439 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a01f3c-47d8-47c9-ae7b-26b77c08e081 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:03:23.570077 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a01f3c-47d8-47c9-ae7b-26b77c08e081 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:03:23.570364 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a01f3c-47d8-47c9-ae7b-26b77c08e081 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:03:23.570642 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a01f3c-47d8-47c9-ae7b-26b77c08e081 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:03:23.640652 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a01f3c-47d8-47c9-ae7b-26b77c08e081 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:03:23.642023 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a01f3c-47d8-47c9-ae7b-26b77c08e081 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:03:23.682800 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:03:23.739613 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 575/1144] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Jul 25 12:03:23 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 25 12:03:23.745449 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65387cdc-1c15-4a3e-9dd2-7c21598ac0c0 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:03:23.841550 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65387cdc-1c15-4a3e-9dd2-7c21598ac0c0 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:03:23.858193 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:03:23.875710 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571fd886-93b1-4823-b7ae-03899bd4ccb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:03:23.876568 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571fd886-93b1-4823-b7ae-03899bd4ccb0 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:03:23.876857 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571fd886-93b1-4823-b7ae-03899bd4ccb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:03:23.877138 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571fd886-93b1-4823-b7ae-03899bd4ccb0 None admin] PATH_INFO: `/v3/projects/b3007e732ccd4085a0ccf9c4ea17f2d6` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:03:23.878311 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571fd886-93b1-4823-b7ae-03899bd4ccb0 None admin] RBAC: Authorizing `identity:get_project(project_id=b3007e732ccd4085a0ccf9c4ea17f2d6)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:03:23.879768 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571fd886-93b1-4823-b7ae-03899bd4ccb0 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:03:23.881456 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 570/1145] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Jul 25 12:03:23 2022] GET /identity/v3/projects/b3007e732ccd4085a0ccf9c4ea17f2d6 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:03:23.887806 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a01f3c-47d8-47c9-ae7b-26b77c08e081 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:03:23.987578 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a01f3c-47d8-47c9-ae7b-26b77c08e081 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:03:24.023184 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238db111-3eb7-42ad-842b-1935cbbb0870 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:03:24.023895 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238db111-3eb7-42ad-842b-1935cbbb0870 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:03:24.024187 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238db111-3eb7-42ad-842b-1935cbbb0870 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:03:24.024469 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238db111-3eb7-42ad-842b-1935cbbb0870 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:03:24.025297 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238db111-3eb7-42ad-842b-1935cbbb0870 None admin] RBAC: Adding query filter params (role.id=a7d20885b6d94928a34b3cc5b3a00006, scope.project.id=b3007e732ccd4085a0ccf9c4ea17f2d6, user.id=831acca9e7ef488b9b3a50f50b045774) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:03:24.025618 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238db111-3eb7-42ad-842b-1935cbbb0870 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:03:24.027049 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238db111-3eb7-42ad-842b-1935cbbb0870 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:03:24.033013 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 576/1146] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Jul 25 12:03:23 2022] GET /identity/v3/role_assignments?user.id=831acca9e7ef488b9b3a50f50b045774&scope.project.id=b3007e732ccd4085a0ccf9c4ea17f2d6&role.id=a7d20885b6d94928a34b3cc5b3a00006 => generated 259 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:03:25.989577 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117f64ae-0456-405e-a739-721dbfcb2799 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:03:25.990081 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117f64ae-0456-405e-a739-721dbfcb2799 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:03:25.990192 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117f64ae-0456-405e-a739-721dbfcb2799 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:03:25.991510 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 571/1147] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 25 12:03:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:03:25.996727 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543342f3-8595-40be-bd97-35ae4ee5595f None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:03:25.997106 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543342f3-8595-40be-bd97-35ae4ee5595f None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:03:25.997291 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543342f3-8595-40be-bd97-35ae4ee5595f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:03:26.010588 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:03:26.016506 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-543342f3-8595-40be-bd97-35ae4ee5595f None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:03:26.017697 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-543342f3-8595-40be-bd97-35ae4ee5595f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:03:26.082604 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 577/1148] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 25 12:03:25 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 12:03:26.104261 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117f64ae-0456-405e-a739-721dbfcb2799 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:03:26.202647 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117f64ae-0456-405e-a739-721dbfcb2799 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:03:26.235033 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256e7e34-af01-4f59-a946-979897c04952 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:03:26.235828 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256e7e34-af01-4f59-a946-979897c04952 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:03:26.236117 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256e7e34-af01-4f59-a946-979897c04952 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:03:26.236401 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256e7e34-af01-4f59-a946-979897c04952 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:03:26.246150 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256e7e34-af01-4f59-a946-979897c04952 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 12:03:26.254159 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-256e7e34-af01-4f59-a946-979897c04952 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 12:03:26.254159 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:03:26.254159 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:03:26.254159 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:03:26.254159 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:03:26.254159 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:03:26.254159 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:03:26.254159 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:03:26.254159 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:03:26.254159 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:03:26.254159 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:03:26.254159 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:03:26.254159 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 12:03:26.254159 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 12:03:26.254159 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 12:03:26.254159 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 12:03:26.255256 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 12:03:26.255256 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 12:03:26.255256 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 12:03:26.255256 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 12:03:26.255256 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:03:26.255256 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:03:26.255256 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:03:26.255256 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:03:26.255256 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:03:26.255256 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:03:26.255256 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:03:26.255256 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 25 12:03:26.255256 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:03:26.255256 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:03:26.255256 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:03:26.255256 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:03:26.256322 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:03:26.256322 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:03:26.256322 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:03:26.256322 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:03:26.256322 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 12:03:26.256322 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 12:03:26.256322 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 12:03:26.256322 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 12:03:26.256322 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 12:03:26.256322 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 12:03:26.256322 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 12:03:26.256322 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 25 12:03:26.256322 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 572/1149] 10.222.0.42 () {56 vars in 1096 bytes} [Mon Jul 25 12:03:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:03:26.261026 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543342f3-8595-40be-bd97-35ae4ee5595f None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:03:26.357121 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543342f3-8595-40be-bd97-35ae4ee5595f None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:03:26.390392 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194d155a-35ce-4e71-998b-74499feff95d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:03:26.391050 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194d155a-35ce-4e71-998b-74499feff95d None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:03:26.391338 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194d155a-35ce-4e71-998b-74499feff95d None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:03:26.391652 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194d155a-35ce-4e71-998b-74499feff95d None admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:03:26.392396 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194d155a-35ce-4e71-998b-74499feff95d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:03:26.392709 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194d155a-35ce-4e71-998b-74499feff95d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:03:26.394072 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194d155a-35ce-4e71-998b-74499feff95d None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:03:26.396234 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:03:26.402221 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 578/1150] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Jul 25 12:03:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:03:26.407860 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256e7e34-af01-4f59-a946-979897c04952 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:03:26.452904 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:03:26.506054 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256e7e34-af01-4f59-a946-979897c04952 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:03:26.538478 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462f09fb-1fe3-47b9-ad32-739100a023af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:03:26.539368 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462f09fb-1fe3-47b9-ad32-739100a023af None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:03:26.539935 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462f09fb-1fe3-47b9-ad32-739100a023af None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:03:26.540216 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462f09fb-1fe3-47b9-ad32-739100a023af None admin] PATH_INFO: `/v3/users/831acca9e7ef488b9b3a50f50b045774` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:03:26.541401 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462f09fb-1fe3-47b9-ad32-739100a023af None admin] RBAC: Authorizing `identity:get_user(user_id=831acca9e7ef488b9b3a50f50b045774)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:03:26.542818 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462f09fb-1fe3-47b9-ad32-739100a023af None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:03:26.544480 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 573/1151] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Jul 25 12:03:26 2022] GET /identity/v3/users/831acca9e7ef488b9b3a50f50b045774 => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:03:26.550810 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194d155a-35ce-4e71-998b-74499feff95d None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:03:26.648116 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194d155a-35ce-4e71-998b-74499feff95d None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:03:26.681337 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4c1171-5f36-48b7-b9f5-7e9bb4b0369b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:03:26.682184 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4c1171-5f36-48b7-b9f5-7e9bb4b0369b None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:03:26.682470 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4c1171-5f36-48b7-b9f5-7e9bb4b0369b None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:03:26.682752 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4c1171-5f36-48b7-b9f5-7e9bb4b0369b None admin] PATH_INFO: `/v3/projects/b3007e732ccd4085a0ccf9c4ea17f2d6` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:03:26.683958 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4c1171-5f36-48b7-b9f5-7e9bb4b0369b None admin] RBAC: Authorizing `identity:get_project(project_id=b3007e732ccd4085a0ccf9c4ea17f2d6)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:03:26.685407 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4c1171-5f36-48b7-b9f5-7e9bb4b0369b None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:03:26.687068 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 579/1152] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Jul 25 12:03:26 2022] GET /identity/v3/projects/b3007e732ccd4085a0ccf9c4ea17f2d6 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:03:26.692840 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462f09fb-1fe3-47b9-ad32-739100a023af None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:03:26.787638 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462f09fb-1fe3-47b9-ad32-739100a023af None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:03:26.800652 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:03:26.820987 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77164ea-5344-4232-9d0e-51a2b8580c97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:03:26.821871 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77164ea-5344-4232-9d0e-51a2b8580c97 None admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:03:26.822155 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77164ea-5344-4232-9d0e-51a2b8580c97 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:03:26.822431 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77164ea-5344-4232-9d0e-51a2b8580c97 None admin] PATH_INFO: `/v3/projects/b3007e732ccd4085a0ccf9c4ea17f2d6/users/831acca9e7ef488b9b3a50f50b045774/roles/a7d20885b6d94928a34b3cc5b3a00006` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:03:26.824386 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77164ea-5344-4232-9d0e-51a2b8580c97 None admin] RBAC: Authorizing `identity:create_grant(project_id=b3007e732ccd4085a0ccf9c4ea17f2d6, user_id=831acca9e7ef488b9b3a50f50b045774, role_id=a7d20885b6d94928a34b3cc5b3a00006)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:03:26.825802 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77164ea-5344-4232-9d0e-51a2b8580c97 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:03:26.847769 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 574/1153] 10.222.0.42 () {58 vars in 1439 bytes} [Mon Jul 25 12:03:26 2022] PUT /identity/v3/projects/b3007e732ccd4085a0ccf9c4ea17f2d6/users/831acca9e7ef488b9b3a50f50b045774/roles/a7d20885b6d94928a34b3cc5b3a00006 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:03:28.837493 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b9fa25-ac8e-491e-b4c5-d806223f2228 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:03:28.837963 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b9fa25-ac8e-491e-b4c5-d806223f2228 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:03:28.838049 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b9fa25-ac8e-491e-b4c5-d806223f2228 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:03:28.839417 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 580/1154] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 25 12:03:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:03:28.844460 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9f4f94-1278-456a-b0a9-476265d83ed9 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:03:28.844899 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9f4f94-1278-456a-b0a9-476265d83ed9 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:03:28.845002 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9f4f94-1278-456a-b0a9-476265d83ed9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:03:28.863101 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-0a9f4f94-1278-456a-b0a9-476265d83ed9 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:03:28.864316 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-0a9f4f94-1278-456a-b0a9-476265d83ed9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:03:28.929707 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 575/1155] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 25 12:03:28 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 12:03:28.952532 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b9fa25-ac8e-491e-b4c5-d806223f2228 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:03:29.017763 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:03:29.054291 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b9fa25-ac8e-491e-b4c5-d806223f2228 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:03:29.086652 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d7b0e6-1a39-4eb5-b5f3-b2077079c34d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:03:29.087441 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d7b0e6-1a39-4eb5-b5f3-b2077079c34d None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:03:29.087758 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d7b0e6-1a39-4eb5-b5f3-b2077079c34d None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:03:29.088044 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d7b0e6-1a39-4eb5-b5f3-b2077079c34d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:03:29.098511 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d7b0e6-1a39-4eb5-b5f3-b2077079c34d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 12:03:29.106125 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-e3d7b0e6-1a39-4eb5-b5f3-b2077079c34d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 12:03:29.106125 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:03:29.106125 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:03:29.106125 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:03:29.106125 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:03:29.106125 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:03:29.106125 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:03:29.106125 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:03:29.106125 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:03:29.106125 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:03:29.106125 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:03:29.106125 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:03:29.106125 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 12:03:29.106125 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 12:03:29.106125 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 12:03:29.106125 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 12:03:29.107955 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 12:03:29.107955 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 12:03:29.107955 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 12:03:29.107955 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 12:03:29.107955 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:03:29.107955 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:03:29.107955 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:03:29.107955 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:03:29.107955 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:03:29.107955 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:03:29.107955 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:03:29.107955 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jul 25 12:03:29.107955 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:03:29.107955 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:03:29.107955 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:03:29.107955 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:03:29.109189 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:03:29.109189 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:03:29.109189 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:03:29.109189 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:03:29.109189 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 12:03:29.109189 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 12:03:29.109189 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 12:03:29.109189 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 12:03:29.109189 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 12:03:29.109189 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 12:03:29.109189 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 12:03:29.109189 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jul 25 12:03:29.109189 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 581/1156] 10.222.0.42 () {56 vars in 1096 bytes} [Mon Jul 25 12:03:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:03:29.113829 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9f4f94-1278-456a-b0a9-476265d83ed9 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:03:29.209245 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9f4f94-1278-456a-b0a9-476265d83ed9 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:03:29.210967 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:03:29.248805 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444b2087-6f31-4994-8924-a6710b77987d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:03:29.249475 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444b2087-6f31-4994-8924-a6710b77987d None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:03:29.249768 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444b2087-6f31-4994-8924-a6710b77987d None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:03:29.250055 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444b2087-6f31-4994-8924-a6710b77987d None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:03:29.250809 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444b2087-6f31-4994-8924-a6710b77987d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:03:29.251124 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444b2087-6f31-4994-8924-a6710b77987d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:03:29.252542 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444b2087-6f31-4994-8924-a6710b77987d None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:03:29.260449 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 576/1157] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Jul 25 12:03:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:03:29.266730 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d7b0e6-1a39-4eb5-b5f3-b2077079c34d None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:03:29.365493 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d7b0e6-1a39-4eb5-b5f3-b2077079c34d None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:03:29.390129 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:03:29.398782 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80150f6b-9a56-4021-832c-3648910919d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:03:29.399665 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80150f6b-9a56-4021-832c-3648910919d6 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:03:29.399958 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80150f6b-9a56-4021-832c-3648910919d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:03:29.400236 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80150f6b-9a56-4021-832c-3648910919d6 None admin] PATH_INFO: `/v3/users/831acca9e7ef488b9b3a50f50b045774` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:03:29.401591 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80150f6b-9a56-4021-832c-3648910919d6 None admin] RBAC: Authorizing `identity:get_user(user_id=831acca9e7ef488b9b3a50f50b045774)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:03:29.402825 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80150f6b-9a56-4021-832c-3648910919d6 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:03:29.405054 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 582/1158] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Jul 25 12:03:29 2022] GET /identity/v3/users/831acca9e7ef488b9b3a50f50b045774 => generated 286 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:03:29.410798 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444b2087-6f31-4994-8924-a6710b77987d None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:03:29.505481 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444b2087-6f31-4994-8924-a6710b77987d None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:03:29.548091 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910e050f-a0b9-444c-9ff3-7b95ea678f7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:03:29.548749 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910e050f-a0b9-444c-9ff3-7b95ea678f7f None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:03:29.549037 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910e050f-a0b9-444c-9ff3-7b95ea678f7f None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:03:29.549317 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910e050f-a0b9-444c-9ff3-7b95ea678f7f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:03:29.578902 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:03:29.614903 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910e050f-a0b9-444c-9ff3-7b95ea678f7f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:03:29.616441 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910e050f-a0b9-444c-9ff3-7b95ea678f7f None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:03:29.711425 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 577/1159] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Jul 25 12:03:29 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 25 12:03:29.717344 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80150f6b-9a56-4021-832c-3648910919d6 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:03:29.814832 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80150f6b-9a56-4021-832c-3648910919d6 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:03:29.847420 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a424f661-aff0-4b03-9d05-c12df2cac071 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:03:29.848300 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a424f661-aff0-4b03-9d05-c12df2cac071 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:03:29.848589 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a424f661-aff0-4b03-9d05-c12df2cac071 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:03:29.848868 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a424f661-aff0-4b03-9d05-c12df2cac071 None admin] PATH_INFO: `/v3/projects/b3007e732ccd4085a0ccf9c4ea17f2d6` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:03:29.850060 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a424f661-aff0-4b03-9d05-c12df2cac071 None admin] RBAC: Authorizing `identity:get_project(project_id=b3007e732ccd4085a0ccf9c4ea17f2d6)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:03:29.851483 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a424f661-aff0-4b03-9d05-c12df2cac071 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:03:29.853195 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 583/1160] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Jul 25 12:03:29 2022] GET /identity/v3/projects/b3007e732ccd4085a0ccf9c4ea17f2d6 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:03:29.859895 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910e050f-a0b9-444c-9ff3-7b95ea678f7f None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:03:29.918960 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:03:29.955974 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910e050f-a0b9-444c-9ff3-7b95ea678f7f None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:03:29.989172 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c37c398-fac6-4dea-84e4-b03283735670 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:03:29.989813 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c37c398-fac6-4dea-84e4-b03283735670 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:03:29.990098 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c37c398-fac6-4dea-84e4-b03283735670 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:03:29.990380 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c37c398-fac6-4dea-84e4-b03283735670 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:03:29.991209 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c37c398-fac6-4dea-84e4-b03283735670 None admin] RBAC: Adding query filter params (role.id=a7d20885b6d94928a34b3cc5b3a00006, scope.project.id=b3007e732ccd4085a0ccf9c4ea17f2d6, user.id=831acca9e7ef488b9b3a50f50b045774) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:03:29.991527 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c37c398-fac6-4dea-84e4-b03283735670 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:03:29.992988 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c37c398-fac6-4dea-84e4-b03283735670 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:03:29.998606 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 578/1161] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Jul 25 12:03:29 2022] GET /identity/v3/role_assignments?user.id=831acca9e7ef488b9b3a50f50b045774&scope.project.id=b3007e732ccd4085a0ccf9c4ea17f2d6&role.id=a7d20885b6d94928a34b3cc5b3a00006 => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:03:31.993545 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb43b99-c9aa-44d0-8aee-cae78981709e None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:03:31.994063 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb43b99-c9aa-44d0-8aee-cae78981709e None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:03:31.994169 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb43b99-c9aa-44d0-8aee-cae78981709e None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:03:31.995443 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 584/1162] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 25 12:03:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:03:32.001135 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2027c93d-2c8a-4bac-95e7-d38be5d88eb4 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:03:32.001567 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2027c93d-2c8a-4bac-95e7-d38be5d88eb4 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:03:32.001690 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2027c93d-2c8a-4bac-95e7-d38be5d88eb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:03:32.020036 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-2027c93d-2c8a-4bac-95e7-d38be5d88eb4 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:03:32.021228 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-2027c93d-2c8a-4bac-95e7-d38be5d88eb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:03:32.086416 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 579/1163] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 25 12:03:31 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 12:03:32.108416 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb43b99-c9aa-44d0-8aee-cae78981709e None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:03:32.169348 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:03:32.208553 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb43b99-c9aa-44d0-8aee-cae78981709e None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:03:32.240893 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889b153e-7315-427e-b39f-2d269752ee09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:03:32.241531 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889b153e-7315-427e-b39f-2d269752ee09 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:03:32.241820 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889b153e-7315-427e-b39f-2d269752ee09 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:03:32.242104 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889b153e-7315-427e-b39f-2d269752ee09 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:03:32.307733 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889b153e-7315-427e-b39f-2d269752ee09 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:03:32.309157 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889b153e-7315-427e-b39f-2d269752ee09 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:03:32.368389 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:03:32.408611 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 585/1164] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Jul 25 12:03:32 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 25 12:03:32.415239 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2027c93d-2c8a-4bac-95e7-d38be5d88eb4 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:03:32.470955 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:03:32.510595 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2027c93d-2c8a-4bac-95e7-d38be5d88eb4 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:03:32.544127 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22834fbb-edde-49a7-86d6-0b6b06f487d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:03:32.544919 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22834fbb-edde-49a7-86d6-0b6b06f487d5 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:03:32.545210 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22834fbb-edde-49a7-86d6-0b6b06f487d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:03:32.545563 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22834fbb-edde-49a7-86d6-0b6b06f487d5 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:03:32.552074 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22834fbb-edde-49a7-86d6-0b6b06f487d5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:03:32.553524 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22834fbb-edde-49a7-86d6-0b6b06f487d5 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:03:32.567371 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-22834fbb-edde-49a7-86d6-0b6b06f487d5 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 25 12:03:32.567371 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:03:32.567371 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jul 25 12:03:32.567371 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jul 25 12:03:32.567371 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 12:03:32.567371 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 12:03:32.567371 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 12:03:32.567371 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 12:03:32.567371 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jul 25 12:03:32.567371 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 25 12:03:32.567371 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 12:03:32.567371 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 25 12:03:32.567371 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:03:32.567371 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:03:32.567371 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:03:32.567371 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:03:32.567371 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:03:32.568440 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:03:32.568440 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:03:32.568440 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:03:32.568440 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:03:32.568440 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:03:32.568440 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:03:32.568440 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 12:03:32.568440 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 12:03:32.568440 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 12:03:32.568440 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 12:03:32.568440 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:03:32.568440 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:03:32.568440 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:03:32.568440 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:03:32.568440 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:03:32.568440 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:03:32.569386 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:03:32.569386 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 25 12:03:32.569386 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:03:32.569386 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:03:32.569386 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:03:32.569386 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:03:32.569386 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:03:32.569386 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:03:32.569386 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:03:32.569386 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:03:32.569386 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jul 25 12:03:32.569386 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 12:03:32.569386 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 25 12:03:32.569386 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 25 12:03:32.569386 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 580/1165] 10.222.0.42 () {56 vars in 1117 bytes} [Mon Jul 25 12:03:32 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:03:32.574691 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889b153e-7315-427e-b39f-2d269752ee09 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:03:32.671864 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889b153e-7315-427e-b39f-2d269752ee09 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:03:32.705274 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924ba903-d0ca-46db-8f60-9497ed4b23b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:03:32.705925 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924ba903-d0ca-46db-8f60-9497ed4b23b7 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:03:32.706215 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924ba903-d0ca-46db-8f60-9497ed4b23b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:03:32.706497 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924ba903-d0ca-46db-8f60-9497ed4b23b7 None admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:03:32.707212 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924ba903-d0ca-46db-8f60-9497ed4b23b7 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:03:32.707524 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924ba903-d0ca-46db-8f60-9497ed4b23b7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:03:32.708916 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924ba903-d0ca-46db-8f60-9497ed4b23b7 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:03:32.715911 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 586/1166] 10.222.0.42 () {56 vars in 1115 bytes} [Mon Jul 25 12:03:32 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:03:34.679384 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fba385b-7e3c-470c-b678-f2ee94693344 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:03:34.679946 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fba385b-7e3c-470c-b678-f2ee94693344 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:03:34.680040 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fba385b-7e3c-470c-b678-f2ee94693344 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:03:34.681351 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 581/1167] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 25 12:03:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:03:34.687118 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601cebdf-e9d1-457d-a732-025e06e59043 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:03:34.687563 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601cebdf-e9d1-457d-a732-025e06e59043 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:03:34.687707 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601cebdf-e9d1-457d-a732-025e06e59043 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:03:34.706172 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-601cebdf-e9d1-457d-a732-025e06e59043 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:03:34.707344 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-601cebdf-e9d1-457d-a732-025e06e59043 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:03:34.726623 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:03:34.772597 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 587/1168] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 25 12:03:34 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 12:03:34.794624 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fba385b-7e3c-470c-b678-f2ee94693344 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:03:34.893043 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fba385b-7e3c-470c-b678-f2ee94693344 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:03:34.909556 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:03:34.926794 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfebcc27-4d07-4b42-a136-27bd5e1cb6a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:03:34.927443 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfebcc27-4d07-4b42-a136-27bd5e1cb6a2 None admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:03:34.927756 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfebcc27-4d07-4b42-a136-27bd5e1cb6a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:03:34.928037 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfebcc27-4d07-4b42-a136-27bd5e1cb6a2 None admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:03:34.928778 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfebcc27-4d07-4b42-a136-27bd5e1cb6a2 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:03:34.930181 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfebcc27-4d07-4b42-a136-27bd5e1cb6a2 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:03:34.945815 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 582/1169] 10.222.0.42 () {60 vars in 1138 bytes} [Mon Jul 25 12:03:34 2022] POST /identity/v3/domains => generated 267 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:03:36.939214 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba9cc84-440c-4911-8607-ab521f1d44c8 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:03:36.939880 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba9cc84-440c-4911-8607-ab521f1d44c8 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:03:36.939880 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba9cc84-440c-4911-8607-ab521f1d44c8 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:03:36.941168 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 588/1170] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 25 12:03:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:03:36.946191 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce03bd6-c8c4-410e-8af0-dd3412ef9391 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:03:36.946621 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce03bd6-c8c4-410e-8af0-dd3412ef9391 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:03:36.946761 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce03bd6-c8c4-410e-8af0-dd3412ef9391 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:03:36.965021 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-4ce03bd6-c8c4-410e-8af0-dd3412ef9391 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:03:36.966214 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-4ce03bd6-c8c4-410e-8af0-dd3412ef9391 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:03:37.031953 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 583/1171] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 25 12:03:36 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 12:03:37.054361 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba9cc84-440c-4911-8607-ab521f1d44c8 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:03:37.152274 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba9cc84-440c-4911-8607-ab521f1d44c8 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:03:37.184611 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b39f0b-4148-4a46-8b86-78536729ea1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:03:37.185447 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b39f0b-4148-4a46-8b86-78536729ea1b None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:03:37.185736 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b39f0b-4148-4a46-8b86-78536729ea1b None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:03:37.186517 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b39f0b-4148-4a46-8b86-78536729ea1b None admin] PATH_INFO: `/v3/domains/862ae2734d974160b91c8584f8ba0a7c` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:03:37.196786 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b39f0b-4148-4a46-8b86-78536729ea1b None admin] RBAC: Authorizing `identity:get_domain(domain_id=862ae2734d974160b91c8584f8ba0a7c)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:03:37.198241 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b39f0b-4148-4a46-8b86-78536729ea1b None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:03:37.199923 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 589/1172] 10.222.0.42 () {56 vars in 1183 bytes} [Mon Jul 25 12:03:37 2022] GET /identity/v3/domains/862ae2734d974160b91c8584f8ba0a7c => generated 267 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:03:37.206059 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce03bd6-c8c4-410e-8af0-dd3412ef9391 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:03:37.301103 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce03bd6-c8c4-410e-8af0-dd3412ef9391 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:03:37.309940 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:03:37.335738 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cee0be1-1614-4479-b6cd-50a3c1a84db0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:03:37.336387 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cee0be1-1614-4479-b6cd-50a3c1a84db0 None admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:03:37.336672 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cee0be1-1614-4479-b6cd-50a3c1a84db0 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:03:37.336949 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cee0be1-1614-4479-b6cd-50a3c1a84db0 None admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:03:37.337650 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cee0be1-1614-4479-b6cd-50a3c1a84db0 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:03:37.339062 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cee0be1-1614-4479-b6cd-50a3c1a84db0 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:03:37.364598 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 584/1173] 10.222.0.42 () {60 vars in 1141 bytes} [Mon Jul 25 12:03:37 2022] POST /identity/v3/projects => generated 361 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:03:39.379827 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3bd7cc-50d7-4e31-949b-24c7a368d82a None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:03:39.380350 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3bd7cc-50d7-4e31-949b-24c7a368d82a None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:03:39.380436 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3bd7cc-50d7-4e31-949b-24c7a368d82a None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:03:39.381797 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 590/1174] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 25 12:03:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:03:39.386810 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d53a63f-5fdd-4486-bebd-8adf05fb9a90 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:03:39.387175 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d53a63f-5fdd-4486-bebd-8adf05fb9a90 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:03:39.387380 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d53a63f-5fdd-4486-bebd-8adf05fb9a90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:03:39.405762 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-6d53a63f-5fdd-4486-bebd-8adf05fb9a90 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:03:39.406955 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-6d53a63f-5fdd-4486-bebd-8adf05fb9a90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:03:39.471794 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 585/1175] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 25 12:03:39 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 12:03:39.495792 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3bd7cc-50d7-4e31-949b-24c7a368d82a None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:03:39.515816 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:03:39.600693 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3bd7cc-50d7-4e31-949b-24c7a368d82a None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:03:39.639917 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d07bc06-2861-42d2-9e23-a1008d9af4ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:03:39.640707 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d07bc06-2861-42d2-9e23-a1008d9af4ab None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:03:39.640998 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d07bc06-2861-42d2-9e23-a1008d9af4ab None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:03:39.641285 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d07bc06-2861-42d2-9e23-a1008d9af4ab None admin] PATH_INFO: `/v3/domains/862ae2734d974160b91c8584f8ba0a7c` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:03:39.642469 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d07bc06-2861-42d2-9e23-a1008d9af4ab None admin] RBAC: Authorizing `identity:get_domain(domain_id=862ae2734d974160b91c8584f8ba0a7c)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:03:39.643915 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d07bc06-2861-42d2-9e23-a1008d9af4ab None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:03:39.645522 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 591/1176] 10.222.0.42 () {56 vars in 1183 bytes} [Mon Jul 25 12:03:39 2022] GET /identity/v3/domains/862ae2734d974160b91c8584f8ba0a7c => generated 267 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:03:39.651433 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d53a63f-5fdd-4486-bebd-8adf05fb9a90 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:03:39.738243 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:03:39.749391 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d53a63f-5fdd-4486-bebd-8adf05fb9a90 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:03:39.782222 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014f6a61-ead9-45e1-a8bc-426fe5377237 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:03:39.782899 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014f6a61-ead9-45e1-a8bc-426fe5377237 None admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:03:39.783191 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014f6a61-ead9-45e1-a8bc-426fe5377237 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:03:39.783475 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014f6a61-ead9-45e1-a8bc-426fe5377237 None admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:03:39.784236 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014f6a61-ead9-45e1-a8bc-426fe5377237 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:03:39.785675 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014f6a61-ead9-45e1-a8bc-426fe5377237 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:03:39.806976 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 586/1177] 10.222.0.42 () {60 vars in 1132 bytes} [Mon Jul 25 12:03:39 2022] POST /identity/v3/users => generated 311 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:03:41.835527 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d47f90b-96cc-43eb-b180-f6214484838b None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:03:41.835982 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d47f90b-96cc-43eb-b180-f6214484838b None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:03:41.836129 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d47f90b-96cc-43eb-b180-f6214484838b None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:03:41.837501 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 592/1178] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 25 12:03:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:03:41.843216 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb2bf94-696b-4fed-b071-9b8a2e911b7e None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:03:41.843604 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb2bf94-696b-4fed-b071-9b8a2e911b7e None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:03:41.843813 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb2bf94-696b-4fed-b071-9b8a2e911b7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:03:41.862130 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-2cb2bf94-696b-4fed-b071-9b8a2e911b7e None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:03:41.863433 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-2cb2bf94-696b-4fed-b071-9b8a2e911b7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:03:41.927648 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 587/1179] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 25 12:03:41 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 12:03:41.950125 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d47f90b-96cc-43eb-b180-f6214484838b None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:03:42.034198 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:03:42.050966 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d47f90b-96cc-43eb-b180-f6214484838b None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:03:42.084115 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ace533-f76c-4bdc-a208-fa325c6a5bf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:03:42.084897 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ace533-f76c-4bdc-a208-fa325c6a5bf5 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:03:42.085186 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ace533-f76c-4bdc-a208-fa325c6a5bf5 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:03:42.085468 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ace533-f76c-4bdc-a208-fa325c6a5bf5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:03:42.095988 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ace533-f76c-4bdc-a208-fa325c6a5bf5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 12:03:42.103492 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-b5ace533-f76c-4bdc-a208-fa325c6a5bf5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 12:03:42.103492 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:03:42.103492 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:03:42.103492 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:03:42.103492 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:03:42.103492 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:03:42.103492 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:03:42.103492 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:03:42.103492 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:03:42.103492 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:03:42.103492 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:03:42.103492 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:03:42.103492 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 12:03:42.103492 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 12:03:42.103492 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 12:03:42.103492 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 12:03:42.104547 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 12:03:42.104547 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 12:03:42.104547 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 12:03:42.104547 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 12:03:42.104547 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:03:42.104547 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:03:42.104547 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:03:42.104547 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:03:42.104547 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:03:42.104547 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:03:42.104547 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:03:42.104547 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jul 25 12:03:42.104547 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:03:42.104547 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:03:42.104547 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:03:42.104547 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:03:42.105569 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:03:42.105569 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:03:42.105569 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:03:42.105569 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:03:42.105569 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 12:03:42.105569 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 12:03:42.105569 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 12:03:42.105569 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 12:03:42.105569 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 12:03:42.105569 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 12:03:42.105569 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 12:03:42.105569 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jul 25 12:03:42.105569 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 593/1180] 10.222.0.42 () {56 vars in 1096 bytes} [Mon Jul 25 12:03:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:03:42.110639 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb2bf94-696b-4fed-b071-9b8a2e911b7e None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:03:42.185390 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:03:42.209342 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb2bf94-696b-4fed-b071-9b8a2e911b7e None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:03:42.241611 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bae850-f931-423e-adb0-852e24c7faf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:03:42.242265 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bae850-f931-423e-adb0-852e24c7faf7 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:03:42.242552 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bae850-f931-423e-adb0-852e24c7faf7 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:03:42.242828 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bae850-f931-423e-adb0-852e24c7faf7 None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:03:42.243584 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0bae850-f931-423e-adb0-852e24c7faf7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:03:42.243897 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0bae850-f931-423e-adb0-852e24c7faf7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:03:42.245259 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0bae850-f931-423e-adb0-852e24c7faf7 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:03:42.252949 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 588/1181] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Jul 25 12:03:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:03:42.258638 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ace533-f76c-4bdc-a208-fa325c6a5bf5 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:03:42.356249 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ace533-f76c-4bdc-a208-fa325c6a5bf5 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:03:42.389820 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ebaf05-44c8-4f22-87d5-2d93780a2a96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:03:42.390679 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ebaf05-44c8-4f22-87d5-2d93780a2a96 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:03:42.390967 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ebaf05-44c8-4f22-87d5-2d93780a2a96 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:03:42.391251 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ebaf05-44c8-4f22-87d5-2d93780a2a96 None admin] PATH_INFO: `/v3/users/3bf618c1bff04e619e024528a82f0b29` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:03:42.403951 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ebaf05-44c8-4f22-87d5-2d93780a2a96 None admin] RBAC: Authorizing `identity:get_user(user_id=3bf618c1bff04e619e024528a82f0b29)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:03:42.405411 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ebaf05-44c8-4f22-87d5-2d93780a2a96 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:03:42.407042 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 594/1182] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Jul 25 12:03:42 2022] GET /identity/v3/users/3bf618c1bff04e619e024528a82f0b29 => generated 311 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:03:42.412856 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bae850-f931-423e-adb0-852e24c7faf7 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:03:42.513386 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bae850-f931-423e-adb0-852e24c7faf7 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:03:42.549720 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5ff72b-7733-4b2f-9ca6-2ecec136b048 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:03:42.550370 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5ff72b-7733-4b2f-9ca6-2ecec136b048 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:03:42.550658 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5ff72b-7733-4b2f-9ca6-2ecec136b048 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:03:42.550943 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5ff72b-7733-4b2f-9ca6-2ecec136b048 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:03:42.560283 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:03:42.617590 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5ff72b-7733-4b2f-9ca6-2ecec136b048 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:03:42.618986 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5ff72b-7733-4b2f-9ca6-2ecec136b048 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:03:42.713581 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 589/1183] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Jul 25 12:03:42 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 25 12:03:42.721613 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ebaf05-44c8-4f22-87d5-2d93780a2a96 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:03:42.724564 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:03:42.820393 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ebaf05-44c8-4f22-87d5-2d93780a2a96 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:03:42.853954 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a132066-7dd3-40ea-92db-c076a2e8dbfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:03:42.854831 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a132066-7dd3-40ea-92db-c076a2e8dbfd None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:03:42.855120 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a132066-7dd3-40ea-92db-c076a2e8dbfd None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:03:42.855403 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a132066-7dd3-40ea-92db-c076a2e8dbfd None admin] PATH_INFO: `/v3/projects/fff73e62cb7a451984233d5ad1500d7d` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:03:42.856668 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a132066-7dd3-40ea-92db-c076a2e8dbfd None admin] RBAC: Authorizing `identity:get_project(project_id=fff73e62cb7a451984233d5ad1500d7d)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:03:42.858141 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a132066-7dd3-40ea-92db-c076a2e8dbfd None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:03:42.859886 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 595/1184] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Jul 25 12:03:42 2022] GET /identity/v3/projects/fff73e62cb7a451984233d5ad1500d7d => generated 361 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:03:42.866357 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5ff72b-7733-4b2f-9ca6-2ecec136b048 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:03:42.917148 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:03:42.975787 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5ff72b-7733-4b2f-9ca6-2ecec136b048 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:03:43.007853 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4535a706-dff5-43a3-adc2-a1452c661b88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:03:43.008503 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4535a706-dff5-43a3-adc2-a1452c661b88 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:03:43.008790 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4535a706-dff5-43a3-adc2-a1452c661b88 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:03:43.009072 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4535a706-dff5-43a3-adc2-a1452c661b88 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:03:43.009896 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4535a706-dff5-43a3-adc2-a1452c661b88 None admin] RBAC: Adding query filter params (role.id=a7d20885b6d94928a34b3cc5b3a00006, scope.project.id=fff73e62cb7a451984233d5ad1500d7d, user.id=3bf618c1bff04e619e024528a82f0b29) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:03:43.010209 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4535a706-dff5-43a3-adc2-a1452c661b88 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:03:43.011651 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4535a706-dff5-43a3-adc2-a1452c661b88 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:03:43.017426 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 590/1185] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Jul 25 12:03:42 2022] GET /identity/v3/role_assignments?user.id=3bf618c1bff04e619e024528a82f0b29&scope.project.id=fff73e62cb7a451984233d5ad1500d7d&role.id=a7d20885b6d94928a34b3cc5b3a00006 => generated 259 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:03:45.043148 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb35d534-d626-4356-84f5-d1390afd8be0 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:03:45.043771 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb35d534-d626-4356-84f5-d1390afd8be0 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:03:45.043771 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb35d534-d626-4356-84f5-d1390afd8be0 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:03:45.045640 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 596/1186] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 25 12:03:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:03:45.050881 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe91f59b-b95e-4e01-8248-95309a9d7ad2 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:03:45.051267 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe91f59b-b95e-4e01-8248-95309a9d7ad2 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:03:45.051453 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe91f59b-b95e-4e01-8248-95309a9d7ad2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:03:45.069806 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-fe91f59b-b95e-4e01-8248-95309a9d7ad2 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:03:45.070994 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-fe91f59b-b95e-4e01-8248-95309a9d7ad2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:03:45.142798 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 591/1187] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 25 12:03:45 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 12:03:45.167069 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb35d534-d626-4356-84f5-d1390afd8be0 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:03:45.235194 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:03:45.266850 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb35d534-d626-4356-84f5-d1390afd8be0 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:03:45.299800 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6d4530-2264-4a9f-acc3-ca64d33d9047 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:03:45.300525 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6d4530-2264-4a9f-acc3-ca64d33d9047 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:03:45.300815 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6d4530-2264-4a9f-acc3-ca64d33d9047 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:03:45.301098 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6d4530-2264-4a9f-acc3-ca64d33d9047 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:03:45.311567 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6d4530-2264-4a9f-acc3-ca64d33d9047 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 12:03:45.319088 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-4b6d4530-2264-4a9f-acc3-ca64d33d9047 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 12:03:45.319088 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:03:45.319088 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:03:45.319088 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:03:45.319088 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:03:45.319088 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:03:45.319088 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:03:45.319088 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:03:45.319088 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:03:45.319088 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:03:45.319088 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:03:45.319088 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:03:45.319088 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 12:03:45.319088 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 12:03:45.319088 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 12:03:45.319088 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 12:03:45.320288 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 12:03:45.320288 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 12:03:45.320288 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 12:03:45.320288 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 12:03:45.320288 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:03:45.320288 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:03:45.320288 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:03:45.320288 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:03:45.320288 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:03:45.320288 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:03:45.320288 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:03:45.320288 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jul 25 12:03:45.320288 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:03:45.320288 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:03:45.320288 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:03:45.320288 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:03:45.321232 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:03:45.321232 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:03:45.321232 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:03:45.321232 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:03:45.321232 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 12:03:45.321232 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 12:03:45.321232 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 12:03:45.321232 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 12:03:45.321232 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 12:03:45.321232 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 12:03:45.321232 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 12:03:45.321232 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jul 25 12:03:45.321232 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 597/1188] 10.222.0.42 () {56 vars in 1096 bytes} [Mon Jul 25 12:03:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:03:45.326249 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe91f59b-b95e-4e01-8248-95309a9d7ad2 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:03:45.356160 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:03:45.423162 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe91f59b-b95e-4e01-8248-95309a9d7ad2 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:03:45.455299 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9357c796-902b-4562-ae20-442ce884f0fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:03:45.455983 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9357c796-902b-4562-ae20-442ce884f0fc None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:03:45.456275 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9357c796-902b-4562-ae20-442ce884f0fc None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:03:45.456561 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9357c796-902b-4562-ae20-442ce884f0fc None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:03:45.457311 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9357c796-902b-4562-ae20-442ce884f0fc None admin] RBAC: Adding query filter params (name=admin) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:03:45.457622 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9357c796-902b-4562-ae20-442ce884f0fc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:03:45.458986 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9357c796-902b-4562-ae20-442ce884f0fc None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:03:45.467347 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 592/1189] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Jul 25 12:03:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:03:45.472999 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6d4530-2264-4a9f-acc3-ca64d33d9047 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:03:45.571964 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6d4530-2264-4a9f-acc3-ca64d33d9047 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:03:45.599599 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:03:45.605773 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8c74d7-3a89-4c7e-8d80-6350b6ff0a2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:03:45.606633 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8c74d7-3a89-4c7e-8d80-6350b6ff0a2e None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:03:45.606918 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8c74d7-3a89-4c7e-8d80-6350b6ff0a2e None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:03:45.607819 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8c74d7-3a89-4c7e-8d80-6350b6ff0a2e None admin] PATH_INFO: `/v3/users/3bf618c1bff04e619e024528a82f0b29` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:03:45.609008 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8c74d7-3a89-4c7e-8d80-6350b6ff0a2e None admin] RBAC: Authorizing `identity:get_user(user_id=3bf618c1bff04e619e024528a82f0b29)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:03:45.610429 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8c74d7-3a89-4c7e-8d80-6350b6ff0a2e None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:03:45.612060 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 598/1190] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Jul 25 12:03:45 2022] GET /identity/v3/users/3bf618c1bff04e619e024528a82f0b29 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:03:45.618023 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9357c796-902b-4562-ae20-442ce884f0fc None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:03:45.708300 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:03:45.715293 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9357c796-902b-4562-ae20-442ce884f0fc None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:03:45.748424 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e67d3e-5e14-470c-b902-ea36f1927c5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:03:45.749275 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e67d3e-5e14-470c-b902-ea36f1927c5b None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:03:45.749566 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e67d3e-5e14-470c-b902-ea36f1927c5b None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:03:45.749852 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e67d3e-5e14-470c-b902-ea36f1927c5b None admin] PATH_INFO: `/v3/projects/fff73e62cb7a451984233d5ad1500d7d` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:03:45.751028 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e67d3e-5e14-470c-b902-ea36f1927c5b None admin] RBAC: Authorizing `identity:get_project(project_id=fff73e62cb7a451984233d5ad1500d7d)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:03:45.752483 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e67d3e-5e14-470c-b902-ea36f1927c5b None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:03:45.754162 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 593/1191] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Jul 25 12:03:45 2022] GET /identity/v3/projects/fff73e62cb7a451984233d5ad1500d7d => generated 361 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:03:45.760060 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8c74d7-3a89-4c7e-8d80-6350b6ff0a2e None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:03:45.859510 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8c74d7-3a89-4c7e-8d80-6350b6ff0a2e None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:03:45.895039 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bf7b17-b4c5-45e4-a7f2-3d8670f04465 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:03:45.895944 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bf7b17-b4c5-45e4-a7f2-3d8670f04465 None admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:03:45.896233 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bf7b17-b4c5-45e4-a7f2-3d8670f04465 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:03:45.896517 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bf7b17-b4c5-45e4-a7f2-3d8670f04465 None admin] PATH_INFO: `/v3/projects/fff73e62cb7a451984233d5ad1500d7d/users/3bf618c1bff04e619e024528a82f0b29/roles/a7d20885b6d94928a34b3cc5b3a00006` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:03:45.898476 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bf7b17-b4c5-45e4-a7f2-3d8670f04465 None admin] RBAC: Authorizing `identity:create_grant(project_id=fff73e62cb7a451984233d5ad1500d7d, user_id=3bf618c1bff04e619e024528a82f0b29, role_id=a7d20885b6d94928a34b3cc5b3a00006)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:03:45.899936 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bf7b17-b4c5-45e4-a7f2-3d8670f04465 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:03:45.912761 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 599/1192] 10.222.0.42 () {58 vars in 1439 bytes} [Mon Jul 25 12:03:45 2022] PUT /identity/v3/projects/fff73e62cb7a451984233d5ad1500d7d/users/3bf618c1bff04e619e024528a82f0b29/roles/a7d20885b6d94928a34b3cc5b3a00006 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:03:47.893114 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a925a3-c3d8-40e6-bbb4-37de5782e76a None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:03:47.893649 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a925a3-c3d8-40e6-bbb4-37de5782e76a None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:03:47.893758 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a925a3-c3d8-40e6-bbb4-37de5782e76a None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:03:47.895036 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 594/1193] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 25 12:03:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:03:47.900227 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8928dad9-e98b-460c-a79d-2ce08d8ecd4b None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:03:47.900652 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8928dad9-e98b-460c-a79d-2ce08d8ecd4b None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:03:47.900802 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8928dad9-e98b-460c-a79d-2ce08d8ecd4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:03:47.919264 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-8928dad9-e98b-460c-a79d-2ce08d8ecd4b None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:03:47.921114 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-8928dad9-e98b-460c-a79d-2ce08d8ecd4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:03:47.976858 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:03:47.992199 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 600/1194] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 25 12:03:47 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 12:03:48.015431 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a925a3-c3d8-40e6-bbb4-37de5782e76a None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:03:48.114481 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a925a3-c3d8-40e6-bbb4-37de5782e76a None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:03:48.146465 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72489e5a-30e1-452a-81a9-168cda584755 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:03:48.147239 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72489e5a-30e1-452a-81a9-168cda584755 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:03:48.147524 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72489e5a-30e1-452a-81a9-168cda584755 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:03:48.147835 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72489e5a-30e1-452a-81a9-168cda584755 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:03:48.158602 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72489e5a-30e1-452a-81a9-168cda584755 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 12:03:48.166404 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-72489e5a-30e1-452a-81a9-168cda584755 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 12:03:48.166404 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:03:48.166404 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:03:48.166404 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:03:48.166404 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:03:48.166404 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:03:48.166404 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:03:48.166404 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:03:48.166404 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:03:48.166404 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:03:48.166404 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:03:48.166404 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:03:48.166404 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 12:03:48.166404 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 12:03:48.166404 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 12:03:48.166404 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 12:03:48.167502 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 12:03:48.167502 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 12:03:48.167502 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 12:03:48.167502 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 12:03:48.167502 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:03:48.167502 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:03:48.167502 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:03:48.167502 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:03:48.167502 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:03:48.167502 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:03:48.167502 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:03:48.167502 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 25 12:03:48.167502 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:03:48.167502 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:03:48.167502 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:03:48.167502 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:03:48.168770 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:03:48.168770 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:03:48.168770 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:03:48.168770 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:03:48.168770 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 12:03:48.168770 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 12:03:48.168770 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 12:03:48.168770 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 12:03:48.168770 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 12:03:48.168770 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 12:03:48.168770 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 12:03:48.168770 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 25 12:03:48.168770 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 595/1195] 10.222.0.42 () {56 vars in 1096 bytes} [Mon Jul 25 12:03:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:03:48.173422 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8928dad9-e98b-460c-a79d-2ce08d8ecd4b None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:03:48.269348 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8928dad9-e98b-460c-a79d-2ce08d8ecd4b None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:03:48.302737 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84274cac-4357-4fc6-b004-e131d5311478 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:03:48.303390 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84274cac-4357-4fc6-b004-e131d5311478 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:03:48.303712 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84274cac-4357-4fc6-b004-e131d5311478 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:03:48.303999 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84274cac-4357-4fc6-b004-e131d5311478 None admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:03:48.304743 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84274cac-4357-4fc6-b004-e131d5311478 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:03:48.305057 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84274cac-4357-4fc6-b004-e131d5311478 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:03:48.306419 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84274cac-4357-4fc6-b004-e131d5311478 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:03:48.314269 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 601/1196] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Jul 25 12:03:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:03:48.319873 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72489e5a-30e1-452a-81a9-168cda584755 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:03:48.344327 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:03:48.418201 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72489e5a-30e1-452a-81a9-168cda584755 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:03:48.450568 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192eb85f-5f51-4aa4-a247-fb1c1c3c2d1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:03:48.452018 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192eb85f-5f51-4aa4-a247-fb1c1c3c2d1b None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:03:48.452305 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192eb85f-5f51-4aa4-a247-fb1c1c3c2d1b None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:03:48.452579 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192eb85f-5f51-4aa4-a247-fb1c1c3c2d1b None admin] PATH_INFO: `/v3/users/3bf618c1bff04e619e024528a82f0b29` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:03:48.453765 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192eb85f-5f51-4aa4-a247-fb1c1c3c2d1b None admin] RBAC: Authorizing `identity:get_user(user_id=3bf618c1bff04e619e024528a82f0b29)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:03:48.455206 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192eb85f-5f51-4aa4-a247-fb1c1c3c2d1b None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:03:48.456840 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 596/1197] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Jul 25 12:03:48 2022] GET /identity/v3/users/3bf618c1bff04e619e024528a82f0b29 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:03:48.462554 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84274cac-4357-4fc6-b004-e131d5311478 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:03:48.512287 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:03:48.562636 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84274cac-4357-4fc6-b004-e131d5311478 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:03:48.594816 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54400b4-d334-44d4-8e4b-c675e7110a4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:03:48.595451 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54400b4-d334-44d4-8e4b-c675e7110a4e None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:03:48.595761 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54400b4-d334-44d4-8e4b-c675e7110a4e None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:03:48.596039 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54400b4-d334-44d4-8e4b-c675e7110a4e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:03:48.660775 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54400b4-d334-44d4-8e4b-c675e7110a4e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:03:48.662202 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54400b4-d334-44d4-8e4b-c675e7110a4e None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:03:48.708105 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:03:48.756808 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 602/1198] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Jul 25 12:03:48 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 25 12:03:48.762634 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192eb85f-5f51-4aa4-a247-fb1c1c3c2d1b None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:03:48.850658 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:03:48.861437 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192eb85f-5f51-4aa4-a247-fb1c1c3c2d1b None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:03:48.894956 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791d4967-556f-43b9-b480-0ebe7ef7a6d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:03:48.895835 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791d4967-556f-43b9-b480-0ebe7ef7a6d9 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:03:48.896130 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791d4967-556f-43b9-b480-0ebe7ef7a6d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:03:48.896413 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791d4967-556f-43b9-b480-0ebe7ef7a6d9 None admin] PATH_INFO: `/v3/projects/fff73e62cb7a451984233d5ad1500d7d` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:03:48.897590 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791d4967-556f-43b9-b480-0ebe7ef7a6d9 None admin] RBAC: Authorizing `identity:get_project(project_id=fff73e62cb7a451984233d5ad1500d7d)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:03:48.899013 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791d4967-556f-43b9-b480-0ebe7ef7a6d9 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:03:48.900730 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 597/1199] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Jul 25 12:03:48 2022] GET /identity/v3/projects/fff73e62cb7a451984233d5ad1500d7d => generated 361 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:03:48.907471 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54400b4-d334-44d4-8e4b-c675e7110a4e None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:03:49.002914 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54400b4-d334-44d4-8e4b-c675e7110a4e None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:03:49.036221 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644ac01f-45a0-4027-ad41-aa464f14ce40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:03:49.036861 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644ac01f-45a0-4027-ad41-aa464f14ce40 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:03:49.037145 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644ac01f-45a0-4027-ad41-aa464f14ce40 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:03:49.037423 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644ac01f-45a0-4027-ad41-aa464f14ce40 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:03:49.038248 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644ac01f-45a0-4027-ad41-aa464f14ce40 None admin] RBAC: Adding query filter params (role.id=a7d20885b6d94928a34b3cc5b3a00006, scope.project.id=fff73e62cb7a451984233d5ad1500d7d, user.id=3bf618c1bff04e619e024528a82f0b29) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:03:49.038563 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644ac01f-45a0-4027-ad41-aa464f14ce40 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:03:49.040275 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644ac01f-45a0-4027-ad41-aa464f14ce40 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:03:49.045906 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 603/1200] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Jul 25 12:03:48 2022] GET /identity/v3/role_assignments?user.id=3bf618c1bff04e619e024528a82f0b29&scope.project.id=fff73e62cb7a451984233d5ad1500d7d&role.id=a7d20885b6d94928a34b3cc5b3a00006 => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:03:58.890890 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=21 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:03:59.043676 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=93 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:04:43.957911 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a766cf0c-ca61-4fe7-94ce-0b8b2df6f658 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:04:43.958622 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a766cf0c-ca61-4fe7-94ce-0b8b2df6f658 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:04:43.958622 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a766cf0c-ca61-4fe7-94ce-0b8b2df6f658 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:04:43.960302 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 598/1201] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 25 12:04:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:04:43.966842 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f881c4-39da-4df8-a657-cc7dc45ff33c None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:04:43.967285 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f881c4-39da-4df8-a657-cc7dc45ff33c None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:04:43.967400 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f881c4-39da-4df8-a657-cc7dc45ff33c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:04:43.998184 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-76f881c4-39da-4df8-a657-cc7dc45ff33c None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:04:43.999488 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-76f881c4-39da-4df8-a657-cc7dc45ff33c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:04:44.090934 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 604/1202] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 25 12:04:43 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 12:04:44.127728 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a766cf0c-ca61-4fe7-94ce-0b8b2df6f658 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:04:44.271313 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a766cf0c-ca61-4fe7-94ce-0b8b2df6f658 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:04:44.323656 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ac6f5f-d9a5-41b2-bb6b-4f0b8c3768ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:04:44.324996 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ac6f5f-d9a5-41b2-bb6b-4f0b8c3768ac None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:04:44.325427 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ac6f5f-d9a5-41b2-bb6b-4f0b8c3768ac None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:04:44.325856 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ac6f5f-d9a5-41b2-bb6b-4f0b8c3768ac None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:04:44.339651 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ac6f5f-d9a5-41b2-bb6b-4f0b8c3768ac None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:04:44.341144 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ac6f5f-d9a5-41b2-bb6b-4f0b8c3768ac None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:04:44.359564 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-78ac6f5f-d9a5-41b2-bb6b-4f0b8c3768ac None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:04:44.359564 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:04:44.359564 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:04:44.359564 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:04:44.359564 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:04:44.359564 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:04:44.359564 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:04:44.359564 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:04:44.359564 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:04:44.359564 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:04:44.359564 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:04:44.359564 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:04:44.359564 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 12:04:44.359564 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 12:04:44.359564 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 12:04:44.360606 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 12:04:44.360606 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:04:44.360606 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:04:44.360606 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:04:44.360606 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:04:44.360606 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:04:44.360606 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:04:44.360606 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:04:44.360606 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 25 12:04:44.360606 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:04:44.360606 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:04:44.360606 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:04:44.360606 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:04:44.360606 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:04:44.360606 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:04:44.360606 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:04:44.360606 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:04:44.364630 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 12:04:44.364630 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 12:04:44.364630 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:04:44.364630 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 25 12:04:44.364630 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 599/1203] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jul 25 12:04:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:04:44.366842 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f881c4-39da-4df8-a657-cc7dc45ff33c None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:04:44.506061 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f881c4-39da-4df8-a657-cc7dc45ff33c None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:04:44.543167 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:04:44.549754 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b9fa88-528c-4c0c-a7c4-c70e020e3957 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:04:44.550691 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b9fa88-528c-4c0c-a7c4-c70e020e3957 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:04:44.551206 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b9fa88-528c-4c0c-a7c4-c70e020e3957 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:04:44.551888 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b9fa88-528c-4c0c-a7c4-c70e020e3957 None admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:04:44.552631 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b9fa88-528c-4c0c-a7c4-c70e020e3957 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:04:44.552965 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b9fa88-528c-4c0c-a7c4-c70e020e3957 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:04:44.554335 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b9fa88-528c-4c0c-a7c4-c70e020e3957 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:04:44.574371 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 605/1204] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jul 25 12:04:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 12:04:44.577393 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ac6f5f-d9a5-41b2-bb6b-4f0b8c3768ac None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:04:44.655868 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:04:44.717681 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ac6f5f-d9a5-41b2-bb6b-4f0b8c3768ac None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:04:44.756127 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfb78c9-e95c-49e2-abc4-64c42b9e0aae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:04:44.756956 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfb78c9-e95c-49e2-abc4-64c42b9e0aae None admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:04:44.757345 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfb78c9-e95c-49e2-abc4-64c42b9e0aae None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:04:44.757765 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfb78c9-e95c-49e2-abc4-64c42b9e0aae None admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:04:44.761864 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfb78c9-e95c-49e2-abc4-64c42b9e0aae None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:04:44.761864 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfb78c9-e95c-49e2-abc4-64c42b9e0aae None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:04:44.803681 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 600/1205] 10.222.0.42 () {60 vars in 1132 bytes} [Mon Jul 25 12:04:44 2022] POST /identity/v3/users => generated 251 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:04:47.061151 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d664bfed-9970-434d-bda7-71d5890e9796 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:04:47.061803 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d664bfed-9970-434d-bda7-71d5890e9796 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:04:47.061803 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d664bfed-9970-434d-bda7-71d5890e9796 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:04:47.063198 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 606/1206] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 25 12:04:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:04:47.069176 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8bd309-f3fc-41cc-83d6-612a716f6d5a None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:04:47.069610 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8bd309-f3fc-41cc-83d6-612a716f6d5a None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:04:47.069776 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8bd309-f3fc-41cc-83d6-612a716f6d5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:04:47.090259 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-0b8bd309-f3fc-41cc-83d6-612a716f6d5a None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:04:47.091492 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-0b8bd309-f3fc-41cc-83d6-612a716f6d5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:04:47.179568 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 601/1207] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 25 12:04:47 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 25 12:04:47.200009 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d664bfed-9970-434d-bda7-71d5890e9796 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:04:47.327007 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d664bfed-9970-434d-bda7-71d5890e9796 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:04:47.364290 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c5513d-0211-498a-a1e9-9ce535ead88e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:04:47.365224 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c5513d-0211-498a-a1e9-9ce535ead88e None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:04:47.365512 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c5513d-0211-498a-a1e9-9ce535ead88e None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:04:47.365819 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c5513d-0211-498a-a1e9-9ce535ead88e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:04:47.378591 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c5513d-0211-498a-a1e9-9ce535ead88e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 12:04:47.386324 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-24c5513d-0211-498a-a1e9-9ce535ead88e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 25 12:04:47.386324 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:04:47.386324 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:04:47.386324 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:04:47.386324 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:04:47.386324 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:04:47.386324 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:04:47.386324 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:04:47.386324 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:04:47.386324 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:04:47.386324 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:04:47.386324 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:04:47.386324 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 12:04:47.386324 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 12:04:47.386324 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 12:04:47.386324 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 12:04:47.388164 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 12:04:47.388164 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 12:04:47.388164 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 12:04:47.388164 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 12:04:47.388164 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:04:47.388164 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:04:47.388164 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:04:47.388164 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:04:47.388164 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:04:47.388164 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:04:47.388164 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:04:47.388164 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jul 25 12:04:47.388164 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:04:47.388164 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:04:47.388164 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:04:47.388164 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:04:47.389296 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:04:47.389296 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:04:47.389296 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:04:47.389296 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:04:47.389296 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 12:04:47.389296 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 12:04:47.389296 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 12:04:47.389296 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 12:04:47.389296 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 12:04:47.389296 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 12:04:47.389296 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 25 12:04:47.389296 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jul 25 12:04:47.389296 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 607/1208] 10.222.0.42 () {56 vars in 1102 bytes} [Mon Jul 25 12:04:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:04:47.393930 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8bd309-f3fc-41cc-83d6-612a716f6d5a None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:04:47.438834 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:04:47.514594 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8bd309-f3fc-41cc-83d6-612a716f6d5a None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:04:47.559533 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d706a42b-ecda-4efc-98d5-0541aef567c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:04:47.560164 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d706a42b-ecda-4efc-98d5-0541aef567c8 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:04:47.560536 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d706a42b-ecda-4efc-98d5-0541aef567c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:04:47.560716 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d706a42b-ecda-4efc-98d5-0541aef567c8 None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:04:47.561565 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d706a42b-ecda-4efc-98d5-0541aef567c8 None admin] RBAC: Adding query filter params (name=service) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:04:47.562001 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d706a42b-ecda-4efc-98d5-0541aef567c8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:04:47.563478 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d706a42b-ecda-4efc-98d5-0541aef567c8 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:04:47.572379 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 602/1209] 10.222.0.42 () {56 vars in 1103 bytes} [Mon Jul 25 12:04:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:04:47.578519 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c5513d-0211-498a-a1e9-9ce535ead88e None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:04:47.635538 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:04:47.700145 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c5513d-0211-498a-a1e9-9ce535ead88e None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:04:47.741919 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9faf9e81-301f-4035-9ddf-44c7c5692dff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:04:47.742790 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9faf9e81-301f-4035-9ddf-44c7c5692dff None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:04:47.743132 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9faf9e81-301f-4035-9ddf-44c7c5692dff None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:04:47.743392 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9faf9e81-301f-4035-9ddf-44c7c5692dff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:04:47.755019 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9faf9e81-301f-4035-9ddf-44c7c5692dff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:04:47.756613 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9faf9e81-301f-4035-9ddf-44c7c5692dff None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:04:47.775908 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-9faf9e81-301f-4035-9ddf-44c7c5692dff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:04:47.775908 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:04:47.775908 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:04:47.775908 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:04:47.775908 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:04:47.775908 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:04:47.775908 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:04:47.775908 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:04:47.775908 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:04:47.775908 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:04:47.775908 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:04:47.775908 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:04:47.775908 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 12:04:47.775908 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 12:04:47.775908 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 12:04:47.776944 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 12:04:47.776944 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:04:47.776944 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:04:47.776944 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:04:47.776944 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:04:47.776944 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:04:47.776944 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:04:47.776944 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:04:47.776944 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jul 25 12:04:47.776944 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:04:47.776944 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:04:47.776944 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:04:47.776944 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:04:47.776944 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:04:47.776944 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:04:47.776944 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:04:47.776944 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:04:47.778046 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 12:04:47.778046 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 12:04:47.778046 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:04:47.778046 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jul 25 12:04:47.778046 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 608/1210] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jul 25 12:04:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:04:47.783182 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d706a42b-ecda-4efc-98d5-0541aef567c8 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:04:47.899880 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:04:47.901974 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d706a42b-ecda-4efc-98d5-0541aef567c8 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:04:47.944353 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de498be4-70c7-4393-bfb8-17fa59993710 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:04:47.945044 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de498be4-70c7-4393-bfb8-17fa59993710 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:04:47.945388 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de498be4-70c7-4393-bfb8-17fa59993710 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:04:47.945657 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de498be4-70c7-4393-bfb8-17fa59993710 None admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:04:47.946473 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de498be4-70c7-4393-bfb8-17fa59993710 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:04:47.946935 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de498be4-70c7-4393-bfb8-17fa59993710 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:04:47.948375 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de498be4-70c7-4393-bfb8-17fa59993710 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:04:47.963635 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 603/1211] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jul 25 12:04:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:04:47.969141 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9faf9e81-301f-4035-9ddf-44c7c5692dff None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:04:48.094471 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9faf9e81-301f-4035-9ddf-44c7c5692dff None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:04:48.096117 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:04:48.138022 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d68ef2-dc71-4151-8217-5c77912e1174 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:04:48.138943 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d68ef2-dc71-4151-8217-5c77912e1174 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:04:48.139297 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d68ef2-dc71-4151-8217-5c77912e1174 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:04:48.139663 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d68ef2-dc71-4151-8217-5c77912e1174 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:04:48.148480 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d68ef2-dc71-4151-8217-5c77912e1174 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:04:48.150613 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d68ef2-dc71-4151-8217-5c77912e1174 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:04:48.166255 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-65d68ef2-dc71-4151-8217-5c77912e1174 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 25 12:04:48.166255 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:04:48.166255 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:04:48.166255 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:04:48.166255 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:04:48.166255 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:04:48.166255 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:04:48.166255 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:04:48.166255 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:04:48.166255 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:04:48.166255 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:04:48.166255 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:04:48.166255 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 12:04:48.166255 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 12:04:48.166255 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 12:04:48.166255 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 12:04:48.167828 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:04:48.167828 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:04:48.167828 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 12:04:48.167828 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 12:04:48.167828 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 12:04:48.167828 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 12:04:48.167828 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:04:48.167828 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:04:48.167828 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:04:48.167828 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:04:48.167828 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:04:48.167828 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jul 25 12:04:48.167828 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:04:48.167828 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:04:48.167828 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:04:48.167828 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:04:48.167828 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:04:48.167828 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:04:48.168992 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:04:48.168992 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:04:48.168992 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 12:04:48.168992 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 12:04:48.168992 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 12:04:48.168992 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 12:04:48.168992 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 12:04:48.168992 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 12:04:48.168992 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 25 12:04:48.168992 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jul 25 12:04:48.171832 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 609/1212] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jul 25 12:04:47 2022] GET /identity/v3/users/placement => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:04:48.173903 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de498be4-70c7-4393-bfb8-17fa59993710 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:04:48.292115 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de498be4-70c7-4393-bfb8-17fa59993710 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:04:48.330782 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a82283b-1e69-4698-910e-a6392f831b22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:04:48.331593 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a82283b-1e69-4698-910e-a6392f831b22 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:04:48.331893 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a82283b-1e69-4698-910e-a6392f831b22 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:04:48.332196 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a82283b-1e69-4698-910e-a6392f831b22 None admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:04:48.333066 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a82283b-1e69-4698-910e-a6392f831b22 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:04:48.333376 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a82283b-1e69-4698-910e-a6392f831b22 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:04:48.335089 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a82283b-1e69-4698-910e-a6392f831b22 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:04:48.359724 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 604/1213] 10.222.0.42 () {56 vars in 1143 bytes} [Mon Jul 25 12:04:48 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:04:48.362028 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d68ef2-dc71-4151-8217-5c77912e1174 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:04:48.487157 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d68ef2-dc71-4151-8217-5c77912e1174 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:04:48.527532 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce66ed02-cb9f-4957-979f-52b342bd05ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:04:48.528340 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce66ed02-cb9f-4957-979f-52b342bd05ca None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:04:48.528643 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce66ed02-cb9f-4957-979f-52b342bd05ca None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:04:48.529000 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce66ed02-cb9f-4957-979f-52b342bd05ca None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:04:48.574339 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:04:48.621972 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce66ed02-cb9f-4957-979f-52b342bd05ca None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:04:48.621972 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce66ed02-cb9f-4957-979f-52b342bd05ca None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:04:48.756912 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 610/1214] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Jul 25 12:04:48 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 25 12:04:48.763442 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a82283b-1e69-4698-910e-a6392f831b22 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:04:48.799448 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:04:48.893109 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a82283b-1e69-4698-910e-a6392f831b22 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:04:48.934828 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7397607b-345e-4a0c-bbcb-a9ae10f24211 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:04:48.935794 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7397607b-345e-4a0c-bbcb-a9ae10f24211 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:04:48.936169 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7397607b-345e-4a0c-bbcb-a9ae10f24211 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:04:48.936569 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7397607b-345e-4a0c-bbcb-a9ae10f24211 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:04:48.949514 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7397607b-345e-4a0c-bbcb-a9ae10f24211 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:04:48.951205 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7397607b-345e-4a0c-bbcb-a9ae10f24211 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:04:48.971251 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-7397607b-345e-4a0c-bbcb-a9ae10f24211 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:04:48.971251 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:04:48.971251 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:04:48.971251 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:04:48.971251 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:04:48.971251 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:04:48.971251 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:04:48.971251 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:04:48.971251 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:04:48.971251 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:04:48.971251 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:04:48.971251 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:04:48.971251 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 12:04:48.971251 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 12:04:48.971251 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 12:04:48.972850 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 12:04:48.972850 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:04:48.972850 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:04:48.972850 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:04:48.972850 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:04:48.972850 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:04:48.972850 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:04:48.972850 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:04:48.972850 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 25 12:04:48.972850 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:04:48.972850 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:04:48.972850 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:04:48.972850 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:04:48.972850 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:04:48.972850 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:04:48.972850 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:04:48.972850 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:04:48.973821 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 12:04:48.973821 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 12:04:48.973821 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:04:48.973821 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 25 12:04:48.973821 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 605/1215] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jul 25 12:04:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:04:48.978909 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce66ed02-cb9f-4957-979f-52b342bd05ca None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:04:49.047340 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:04:49.088989 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce66ed02-cb9f-4957-979f-52b342bd05ca None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:04:49.126235 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1ddecc-72fb-4039-bcd9-4e6adf5943ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:04:49.127036 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1ddecc-72fb-4039-bcd9-4e6adf5943ba None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:04:49.127435 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1ddecc-72fb-4039-bcd9-4e6adf5943ba None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:04:49.127838 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1ddecc-72fb-4039-bcd9-4e6adf5943ba None admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:04:49.128655 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1ddecc-72fb-4039-bcd9-4e6adf5943ba None admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:04:49.129061 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1ddecc-72fb-4039-bcd9-4e6adf5943ba None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:04:49.130509 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1ddecc-72fb-4039-bcd9-4e6adf5943ba None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:04:49.147214 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 611/1216] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jul 25 12:04:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:04:49.148510 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7397607b-345e-4a0c-bbcb-a9ae10f24211 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:04:49.241234 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:04:49.286442 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7397607b-345e-4a0c-bbcb-a9ae10f24211 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:04:49.322649 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f6fe1d-ac82-4376-a506-3fefc9738e81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:04:49.323372 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f6fe1d-ac82-4376-a506-3fefc9738e81 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:04:49.323753 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f6fe1d-ac82-4376-a506-3fefc9738e81 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:04:49.324035 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f6fe1d-ac82-4376-a506-3fefc9738e81 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:04:49.330653 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f6fe1d-ac82-4376-a506-3fefc9738e81 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:04:49.332145 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f6fe1d-ac82-4376-a506-3fefc9738e81 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:04:49.346150 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-f6f6fe1d-ac82-4376-a506-3fefc9738e81 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 12:04:49.346150 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:04:49.346150 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:04:49.346150 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:04:49.346150 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:04:49.346150 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:04:49.346150 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:04:49.346150 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:04:49.346150 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:04:49.346150 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:04:49.346150 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:04:49.346150 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:04:49.346150 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 12:04:49.346150 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 12:04:49.346150 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 12:04:49.347140 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 12:04:49.347140 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:04:49.347140 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:04:49.347140 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:04:49.347140 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:04:49.347140 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:04:49.347140 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:04:49.347140 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:04:49.347140 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 25 12:04:49.347140 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:04:49.347140 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:04:49.347140 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:04:49.347140 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:04:49.347140 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:04:49.347140 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:04:49.347140 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:04:49.347140 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:04:49.348439 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 12:04:49.348439 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 12:04:49.348439 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 12:04:49.348439 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 12:04:49.348439 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 12:04:49.348439 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 12:04:49.348439 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 12:04:49.348439 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 25 12:04:49.352380 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 606/1217] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Jul 25 12:04:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 12:04:49.354129 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1ddecc-72fb-4039-bcd9-4e6adf5943ba None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:04:49.472114 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1ddecc-72fb-4039-bcd9-4e6adf5943ba None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:04:49.498241 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:04:49.515319 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7d9cfc-a8cb-4687-8fcb-87a5455c0fe8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:04:49.516496 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7d9cfc-a8cb-4687-8fcb-87a5455c0fe8 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:04:49.516496 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7d9cfc-a8cb-4687-8fcb-87a5455c0fe8 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:04:49.516954 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7d9cfc-a8cb-4687-8fcb-87a5455c0fe8 None admin] PATH_INFO: `/v3/projects` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:04:49.517963 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7d9cfc-a8cb-4687-8fcb-87a5455c0fe8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:04:49.518418 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7d9cfc-a8cb-4687-8fcb-87a5455c0fe8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:04:49.520021 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7d9cfc-a8cb-4687-8fcb-87a5455c0fe8 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:04:49.534085 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 612/1218] 10.222.0.42 () {56 vars in 1148 bytes} [Mon Jul 25 12:04:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:04:49.541420 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f6fe1d-ac82-4376-a506-3fefc9738e81 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:04:49.667256 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f6fe1d-ac82-4376-a506-3fefc9738e81 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:04:49.702844 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:04:49.711255 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cfd132-ea18-44e7-8840-98e4e01548a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:04:49.712149 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cfd132-ea18-44e7-8840-98e4e01548a2 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:04:49.712434 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cfd132-ea18-44e7-8840-98e4e01548a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:04:49.713740 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cfd132-ea18-44e7-8840-98e4e01548a2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:04:49.714589 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16cfd132-ea18-44e7-8840-98e4e01548a2 None admin] RBAC: Adding query filter params (role.id=79dec36aee5b4322bf652041c25ca6e4, scope.project.id=b9f62b8f37304b8a9a4bb54205347577, user.id=8012afab6cac4358ab878732e58d0689) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:04:49.714906 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16cfd132-ea18-44e7-8840-98e4e01548a2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:04:49.716379 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16cfd132-ea18-44e7-8840-98e4e01548a2 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:04:49.727848 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 607/1219] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Jul 25 12:04:49 2022] GET /identity/v3/role_assignments?user.id=8012afab6cac4358ab878732e58d0689&scope.project.id=b9f62b8f37304b8a9a4bb54205347577&role.id=79dec36aee5b4322bf652041c25ca6e4 => generated 259 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:04:52.012127 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0973f9f6-a187-4f74-9b84-629e306671bd None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:04:52.012687 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0973f9f6-a187-4f74-9b84-629e306671bd None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:04:52.012687 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0973f9f6-a187-4f74-9b84-629e306671bd None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:04:52.014046 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 613/1220] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 25 12:04:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:04:52.020040 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08153c48-5ada-4005-8b68-d9d18488cb2a None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:04:52.020489 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08153c48-5ada-4005-8b68-d9d18488cb2a None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:04:52.020625 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08153c48-5ada-4005-8b68-d9d18488cb2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:04:52.039510 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-08153c48-5ada-4005-8b68-d9d18488cb2a None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:04:52.040747 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-08153c48-5ada-4005-8b68-d9d18488cb2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:04:52.115016 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 608/1221] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 25 12:04:52 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 25 12:04:52.139809 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0973f9f6-a187-4f74-9b84-629e306671bd None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:04:52.253820 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0973f9f6-a187-4f74-9b84-629e306671bd None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:04:52.291001 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf813ddb-0253-458f-ac72-e6822d688d8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:04:52.291741 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf813ddb-0253-458f-ac72-e6822d688d8e None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:04:52.292029 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf813ddb-0253-458f-ac72-e6822d688d8e None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:04:52.292309 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf813ddb-0253-458f-ac72-e6822d688d8e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:04:52.302662 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf813ddb-0253-458f-ac72-e6822d688d8e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 12:04:52.312344 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-bf813ddb-0253-458f-ac72-e6822d688d8e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 25 12:04:52.312344 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:04:52.312344 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:04:52.312344 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:04:52.312344 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:04:52.312344 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:04:52.312344 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:04:52.312344 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:04:52.312344 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:04:52.312344 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:04:52.312344 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:04:52.312344 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:04:52.312344 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 12:04:52.312344 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 12:04:52.312344 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 12:04:52.312344 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 12:04:52.313307 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 12:04:52.313307 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 12:04:52.313307 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 12:04:52.313307 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 12:04:52.313307 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:04:52.313307 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:04:52.313307 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:04:52.313307 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:04:52.313307 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:04:52.313307 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:04:52.313307 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:04:52.313307 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jul 25 12:04:52.313307 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:04:52.313307 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:04:52.313307 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:04:52.313307 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:04:52.314166 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:04:52.314166 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:04:52.314166 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:04:52.314166 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:04:52.314166 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 12:04:52.314166 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 12:04:52.314166 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 12:04:52.314166 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 12:04:52.314166 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 12:04:52.314166 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 12:04:52.314166 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 25 12:04:52.314166 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jul 25 12:04:52.314166 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 614/1222] 10.222.0.42 () {56 vars in 1102 bytes} [Mon Jul 25 12:04:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 12:04:52.320369 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08153c48-5ada-4005-8b68-d9d18488cb2a None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:04:52.430121 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08153c48-5ada-4005-8b68-d9d18488cb2a None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:04:52.454362 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:04:52.466210 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f243c2-56d3-4925-9ba8-50257b51509e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:04:52.466877 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f243c2-56d3-4925-9ba8-50257b51509e None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:04:52.467166 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f243c2-56d3-4925-9ba8-50257b51509e None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:04:52.467443 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f243c2-56d3-4925-9ba8-50257b51509e None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:04:52.468215 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f243c2-56d3-4925-9ba8-50257b51509e None admin] RBAC: Adding query filter params (name=service) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:04:52.468525 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f243c2-56d3-4925-9ba8-50257b51509e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:04:52.469886 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f243c2-56d3-4925-9ba8-50257b51509e None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:04:52.478692 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 609/1223] 10.222.0.42 () {56 vars in 1103 bytes} [Mon Jul 25 12:04:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:04:52.485203 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf813ddb-0253-458f-ac72-e6822d688d8e None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:04:52.533909 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:04:52.601426 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf813ddb-0253-458f-ac72-e6822d688d8e None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:04:52.638073 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db72552-5056-4ef9-a571-9a3de14434c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:04:52.639016 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db72552-5056-4ef9-a571-9a3de14434c5 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:04:52.639321 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db72552-5056-4ef9-a571-9a3de14434c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:04:52.639644 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db72552-5056-4ef9-a571-9a3de14434c5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:04:52.652158 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db72552-5056-4ef9-a571-9a3de14434c5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:04:52.653790 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db72552-5056-4ef9-a571-9a3de14434c5 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:04:52.671217 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-4db72552-5056-4ef9-a571-9a3de14434c5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:04:52.671217 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:04:52.671217 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:04:52.671217 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:04:52.671217 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:04:52.671217 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:04:52.671217 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:04:52.671217 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:04:52.671217 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:04:52.671217 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:04:52.671217 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:04:52.671217 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:04:52.671217 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 12:04:52.671217 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 12:04:52.671217 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 12:04:52.672366 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 12:04:52.672366 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:04:52.672366 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:04:52.672366 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:04:52.672366 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:04:52.672366 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:04:52.672366 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:04:52.672366 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:04:52.672366 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jul 25 12:04:52.672366 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:04:52.672366 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:04:52.672366 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:04:52.672366 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:04:52.672366 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:04:52.672366 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:04:52.672366 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:04:52.672366 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:04:52.673292 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 12:04:52.673292 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 12:04:52.673292 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:04:52.673292 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jul 25 12:04:52.673292 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 615/1224] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jul 25 12:04:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:04:52.678378 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f243c2-56d3-4925-9ba8-50257b51509e None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:04:52.794063 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f243c2-56d3-4925-9ba8-50257b51509e None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:04:52.831315 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62c77f2-8c71-4a03-8f7b-f827a6a1a0c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:04:52.832021 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62c77f2-8c71-4a03-8f7b-f827a6a1a0c6 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:04:52.832312 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62c77f2-8c71-4a03-8f7b-f827a6a1a0c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:04:52.832593 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62c77f2-8c71-4a03-8f7b-f827a6a1a0c6 None admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:04:52.833326 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62c77f2-8c71-4a03-8f7b-f827a6a1a0c6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:04:52.833641 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62c77f2-8c71-4a03-8f7b-f827a6a1a0c6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:04:52.835012 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62c77f2-8c71-4a03-8f7b-f827a6a1a0c6 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:04:52.847394 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 610/1225] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jul 25 12:04:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:04:52.853341 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db72552-5056-4ef9-a571-9a3de14434c5 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:04:52.953848 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:04:52.961837 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db72552-5056-4ef9-a571-9a3de14434c5 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:04:52.999863 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f383aeb4-ccb1-4d61-b3b9-7789d87de0c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:04:53.000696 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f383aeb4-ccb1-4d61-b3b9-7789d87de0c7 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:04:53.001027 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f383aeb4-ccb1-4d61-b3b9-7789d87de0c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:04:53.001380 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f383aeb4-ccb1-4d61-b3b9-7789d87de0c7 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:04:53.010665 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f383aeb4-ccb1-4d61-b3b9-7789d87de0c7 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:04:53.012446 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f383aeb4-ccb1-4d61-b3b9-7789d87de0c7 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:04:53.027044 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-f383aeb4-ccb1-4d61-b3b9-7789d87de0c7 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 25 12:04:53.027044 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:04:53.027044 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:04:53.027044 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:04:53.027044 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:04:53.027044 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:04:53.027044 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:04:53.027044 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:04:53.027044 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:04:53.027044 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:04:53.027044 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:04:53.027044 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:04:53.027044 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 12:04:53.027044 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 12:04:53.027044 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 12:04:53.027044 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 12:04:53.028300 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:04:53.028300 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:04:53.028300 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 12:04:53.028300 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 12:04:53.028300 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 12:04:53.028300 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 12:04:53.028300 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:04:53.028300 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:04:53.028300 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:04:53.028300 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:04:53.028300 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:04:53.028300 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jul 25 12:04:53.028300 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:04:53.028300 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:04:53.028300 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:04:53.028300 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:04:53.028300 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:04:53.028300 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:04:53.034918 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:04:53.034918 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:04:53.034918 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 12:04:53.034918 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 12:04:53.034918 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 12:04:53.034918 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 12:04:53.034918 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 12:04:53.034918 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 12:04:53.034918 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 25 12:04:53.034918 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jul 25 12:04:53.034918 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 616/1226] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jul 25 12:04:52 2022] GET /identity/v3/users/placement => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:04:53.035626 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62c77f2-8c71-4a03-8f7b-f827a6a1a0c6 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:04:53.087506 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:04:53.146727 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62c77f2-8c71-4a03-8f7b-f827a6a1a0c6 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:04:53.185706 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33ad8bb-ba80-4126-ba42-af6016de3deb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:04:53.186476 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33ad8bb-ba80-4126-ba42-af6016de3deb None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:04:53.186820 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33ad8bb-ba80-4126-ba42-af6016de3deb None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:04:53.187128 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33ad8bb-ba80-4126-ba42-af6016de3deb None admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:04:53.188198 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33ad8bb-ba80-4126-ba42-af6016de3deb None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:04:53.188550 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33ad8bb-ba80-4126-ba42-af6016de3deb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:04:53.190397 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33ad8bb-ba80-4126-ba42-af6016de3deb None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:04:53.210007 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 611/1227] 10.222.0.42 () {56 vars in 1143 bytes} [Mon Jul 25 12:04:53 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 12:04:53.216335 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f383aeb4-ccb1-4d61-b3b9-7789d87de0c7 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:04:53.327823 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f383aeb4-ccb1-4d61-b3b9-7789d87de0c7 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:04:53.364991 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a34bc5-54ec-4b86-abd3-ffaca7a96c1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:04:53.365889 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a34bc5-54ec-4b86-abd3-ffaca7a96c1e None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:04:53.366219 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a34bc5-54ec-4b86-abd3-ffaca7a96c1e None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:04:53.366466 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a34bc5-54ec-4b86-abd3-ffaca7a96c1e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:04:53.378737 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a34bc5-54ec-4b86-abd3-ffaca7a96c1e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:04:53.380454 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a34bc5-54ec-4b86-abd3-ffaca7a96c1e None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:04:53.397620 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-92a34bc5-54ec-4b86-abd3-ffaca7a96c1e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:04:53.397620 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:04:53.397620 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:04:53.397620 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:04:53.397620 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:04:53.397620 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:04:53.397620 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:04:53.397620 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:04:53.397620 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:04:53.397620 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:04:53.397620 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:04:53.397620 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:04:53.397620 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 12:04:53.397620 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 12:04:53.397620 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 12:04:53.404244 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 12:04:53.404244 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:04:53.404244 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:04:53.404244 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:04:53.404244 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:04:53.404244 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:04:53.404244 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:04:53.404244 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:04:53.404244 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jul 25 12:04:53.404244 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:04:53.404244 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:04:53.404244 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:04:53.404244 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:04:53.404244 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:04:53.404244 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:04:53.404244 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:04:53.404244 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:04:53.405394 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 12:04:53.405394 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 12:04:53.405394 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:04:53.405394 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jul 25 12:04:53.405394 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 617/1228] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jul 25 12:04:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 12:04:53.405742 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33ad8bb-ba80-4126-ba42-af6016de3deb None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:04:53.515850 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33ad8bb-ba80-4126-ba42-af6016de3deb None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:04:53.518155 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:04:53.556713 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2230cb7-d940-486f-915a-c8b024322929 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:04:53.557449 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2230cb7-d940-486f-915a-c8b024322929 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:04:53.557782 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2230cb7-d940-486f-915a-c8b024322929 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:04:53.558187 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2230cb7-d940-486f-915a-c8b024322929 None admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:04:53.558948 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2230cb7-d940-486f-915a-c8b024322929 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:04:53.559335 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2230cb7-d940-486f-915a-c8b024322929 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:04:53.561101 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2230cb7-d940-486f-915a-c8b024322929 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:04:53.574190 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 612/1229] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jul 25 12:04:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:04:53.580175 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a34bc5-54ec-4b86-abd3-ffaca7a96c1e None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:04:53.590875 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:04:53.694454 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a34bc5-54ec-4b86-abd3-ffaca7a96c1e None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:04:53.731396 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e4b2a1-096e-4158-b5bf-071a181bef86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:04:53.732226 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e4b2a1-096e-4158-b5bf-071a181bef86 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:04:53.732516 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e4b2a1-096e-4158-b5bf-071a181bef86 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:04:53.732801 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e4b2a1-096e-4158-b5bf-071a181bef86 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:04:53.739492 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e4b2a1-096e-4158-b5bf-071a181bef86 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:04:53.740952 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e4b2a1-096e-4158-b5bf-071a181bef86 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:04:53.755474 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-a8e4b2a1-096e-4158-b5bf-071a181bef86 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 12:04:53.755474 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:04:53.755474 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:04:53.755474 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:04:53.755474 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:04:53.755474 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:04:53.755474 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:04:53.755474 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:04:53.755474 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:04:53.755474 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:04:53.755474 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:04:53.755474 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:04:53.755474 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 12:04:53.755474 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 12:04:53.755474 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 12:04:53.756610 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 12:04:53.756610 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:04:53.756610 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:04:53.756610 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:04:53.756610 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:04:53.756610 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:04:53.756610 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:04:53.756610 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:04:53.756610 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jul 25 12:04:53.756610 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:04:53.756610 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:04:53.756610 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:04:53.756610 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:04:53.756610 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:04:53.756610 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:04:53.756610 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:04:53.756610 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:04:53.757707 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 12:04:53.757707 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 12:04:53.757707 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 12:04:53.757707 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 12:04:53.757707 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 12:04:53.757707 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 12:04:53.757707 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 12:04:53.757707 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jul 25 12:04:53.757707 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 618/1230] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Jul 25 12:04:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:04:53.763958 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2230cb7-d940-486f-915a-c8b024322929 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:04:53.877465 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2230cb7-d940-486f-915a-c8b024322929 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:04:53.914417 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f2d6fd-3ecc-45b2-be2b-31c6d5b53326 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:04:53.915167 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f2d6fd-3ecc-45b2-be2b-31c6d5b53326 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:04:53.915472 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f2d6fd-3ecc-45b2-be2b-31c6d5b53326 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:04:53.915847 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f2d6fd-3ecc-45b2-be2b-31c6d5b53326 None admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:04:53.916732 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f2d6fd-3ecc-45b2-be2b-31c6d5b53326 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:04:53.917104 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f2d6fd-3ecc-45b2-be2b-31c6d5b53326 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:04:53.918621 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f2d6fd-3ecc-45b2-be2b-31c6d5b53326 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:04:53.932116 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 613/1231] 10.222.0.42 () {56 vars in 1148 bytes} [Mon Jul 25 12:04:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:04:53.938388 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e4b2a1-096e-4158-b5bf-071a181bef86 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:04:54.011914 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:04:54.051277 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e4b2a1-096e-4158-b5bf-071a181bef86 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:04:54.088713 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca68555b-2d83-4020-81cf-21043aca07e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:04:54.089760 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca68555b-2d83-4020-81cf-21043aca07e9 None admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:04:54.090050 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca68555b-2d83-4020-81cf-21043aca07e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:04:54.090420 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca68555b-2d83-4020-81cf-21043aca07e9 None admin] PATH_INFO: `/v3/projects/b9f62b8f37304b8a9a4bb54205347577/users/8012afab6cac4358ab878732e58d0689/roles/79dec36aee5b4322bf652041c25ca6e4` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:04:54.107013 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca68555b-2d83-4020-81cf-21043aca07e9 None admin] RBAC: Authorizing `identity:create_grant(project_id=b9f62b8f37304b8a9a4bb54205347577, user_id=8012afab6cac4358ab878732e58d0689, role_id=79dec36aee5b4322bf652041c25ca6e4)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:04:54.108652 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca68555b-2d83-4020-81cf-21043aca07e9 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:04:54.125828 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 619/1232] 10.222.0.42 () {58 vars in 1439 bytes} [Mon Jul 25 12:04:53 2022] PUT /identity/v3/projects/b9f62b8f37304b8a9a4bb54205347577/users/8012afab6cac4358ab878732e58d0689/roles/79dec36aee5b4322bf652041c25ca6e4 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 12:04:56.207836 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823d9b54-9431-40d1-8579-bbfbbe03e4af None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:04:56.208421 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823d9b54-9431-40d1-8579-bbfbbe03e4af None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:04:56.208421 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823d9b54-9431-40d1-8579-bbfbbe03e4af None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:04:56.210029 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 614/1233] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 25 12:04:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:04:56.215785 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae04070-3b55-45fc-a50a-45359beb2bd6 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:04:56.216358 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae04070-3b55-45fc-a50a-45359beb2bd6 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:04:56.216521 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae04070-3b55-45fc-a50a-45359beb2bd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:04:56.235642 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-6ae04070-3b55-45fc-a50a-45359beb2bd6 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:04:56.236869 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-6ae04070-3b55-45fc-a50a-45359beb2bd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:04:56.308914 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 620/1234] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 25 12:04:56 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 12:04:56.332414 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823d9b54-9431-40d1-8579-bbfbbe03e4af None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:04:56.364205 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:04:56.444135 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823d9b54-9431-40d1-8579-bbfbbe03e4af None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:04:56.481450 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6824cb-0a17-406a-98e3-dfd324809886 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:04:56.482210 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6824cb-0a17-406a-98e3-dfd324809886 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:04:56.482561 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6824cb-0a17-406a-98e3-dfd324809886 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:04:56.482842 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6824cb-0a17-406a-98e3-dfd324809886 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:04:56.493897 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6824cb-0a17-406a-98e3-dfd324809886 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 12:04:56.501579 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-bf6824cb-0a17-406a-98e3-dfd324809886 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 25 12:04:56.501579 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:04:56.501579 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:04:56.501579 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:04:56.501579 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:04:56.501579 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:04:56.501579 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:04:56.501579 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:04:56.501579 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:04:56.501579 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:04:56.501579 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:04:56.501579 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:04:56.501579 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 12:04:56.501579 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 12:04:56.501579 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 12:04:56.501579 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 12:04:56.502653 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 12:04:56.502653 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 12:04:56.502653 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 12:04:56.502653 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 12:04:56.502653 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:04:56.502653 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:04:56.502653 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:04:56.502653 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:04:56.502653 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:04:56.502653 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:04:56.502653 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:04:56.502653 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 25 12:04:56.502653 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:04:56.502653 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:04:56.502653 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:04:56.502653 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:04:56.503716 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:04:56.503716 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:04:56.503716 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:04:56.503716 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:04:56.503716 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 12:04:56.503716 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 12:04:56.503716 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 12:04:56.503716 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 12:04:56.503716 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 12:04:56.503716 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 12:04:56.503716 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 25 12:04:56.503716 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 25 12:04:56.503716 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 615/1235] 10.222.0.42 () {56 vars in 1102 bytes} [Mon Jul 25 12:04:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:04:56.509243 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae04070-3b55-45fc-a50a-45359beb2bd6 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:04:56.563917 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:04:56.627405 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae04070-3b55-45fc-a50a-45359beb2bd6 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:04:56.663564 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa6a71d-0e49-4e93-8c48-7586824a5167 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:04:56.665214 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa6a71d-0e49-4e93-8c48-7586824a5167 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:04:56.665668 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa6a71d-0e49-4e93-8c48-7586824a5167 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:04:56.666091 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa6a71d-0e49-4e93-8c48-7586824a5167 None admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:04:56.667144 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa6a71d-0e49-4e93-8c48-7586824a5167 None admin] RBAC: Adding query filter params (name=service) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:04:56.667513 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa6a71d-0e49-4e93-8c48-7586824a5167 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:04:56.669508 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa6a71d-0e49-4e93-8c48-7586824a5167 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:04:56.678031 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 621/1236] 10.222.0.42 () {56 vars in 1103 bytes} [Mon Jul 25 12:04:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:04:56.685094 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6824cb-0a17-406a-98e3-dfd324809886 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:04:56.783269 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:04:56.803525 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6824cb-0a17-406a-98e3-dfd324809886 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:04:56.841713 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65995e33-8d82-4619-9678-bdc72c8fa8d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:04:56.842550 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65995e33-8d82-4619-9678-bdc72c8fa8d2 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:04:56.842852 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65995e33-8d82-4619-9678-bdc72c8fa8d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:04:56.843142 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65995e33-8d82-4619-9678-bdc72c8fa8d2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:04:56.855520 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65995e33-8d82-4619-9678-bdc72c8fa8d2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:04:56.857025 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65995e33-8d82-4619-9678-bdc72c8fa8d2 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:04:56.875036 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-65995e33-8d82-4619-9678-bdc72c8fa8d2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:04:56.875036 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:04:56.875036 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:04:56.875036 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:04:56.875036 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:04:56.875036 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:04:56.875036 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:04:56.875036 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:04:56.875036 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:04:56.875036 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:04:56.875036 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:04:56.875036 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:04:56.875036 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 12:04:56.875036 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 12:04:56.875036 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 12:04:56.876222 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 12:04:56.876222 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:04:56.876222 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:04:56.876222 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:04:56.876222 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:04:56.876222 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:04:56.876222 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:04:56.876222 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:04:56.876222 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 25 12:04:56.876222 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:04:56.876222 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:04:56.876222 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:04:56.876222 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:04:56.876222 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:04:56.876222 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:04:56.876222 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:04:56.876222 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:04:56.877331 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 12:04:56.877331 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 12:04:56.877331 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:04:56.877331 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 25 12:04:56.877331 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 616/1237] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jul 25 12:04:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:04:56.882884 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa6a71d-0e49-4e93-8c48-7586824a5167 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:04:56.991705 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa6a71d-0e49-4e93-8c48-7586824a5167 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:04:57.000994 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:04:57.030375 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2730ff58-d592-4f26-979e-b9acb07b81b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:04:57.030981 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2730ff58-d592-4f26-979e-b9acb07b81b9 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:04:57.031267 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2730ff58-d592-4f26-979e-b9acb07b81b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:04:57.031573 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2730ff58-d592-4f26-979e-b9acb07b81b9 None admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:04:57.032304 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2730ff58-d592-4f26-979e-b9acb07b81b9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:04:57.032619 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2730ff58-d592-4f26-979e-b9acb07b81b9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:04:57.033986 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2730ff58-d592-4f26-979e-b9acb07b81b9 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:04:57.049703 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 622/1238] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jul 25 12:04:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:04:57.056696 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65995e33-8d82-4619-9678-bdc72c8fa8d2 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:04:57.167730 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65995e33-8d82-4619-9678-bdc72c8fa8d2 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:04:57.204781 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc1ba7e-3e4b-4bce-a2c5-e12999c5518b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:04:57.205620 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc1ba7e-3e4b-4bce-a2c5-e12999c5518b None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:04:57.205874 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc1ba7e-3e4b-4bce-a2c5-e12999c5518b None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:04:57.206154 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc1ba7e-3e4b-4bce-a2c5-e12999c5518b None admin] PATH_INFO: `/v3/users/placement` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:04:57.209255 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:04:57.217087 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc1ba7e-3e4b-4bce-a2c5-e12999c5518b None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:04:57.219292 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc1ba7e-3e4b-4bce-a2c5-e12999c5518b None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:04:57.237040 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-3dc1ba7e-3e4b-4bce-a2c5-e12999c5518b None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 25 12:04:57.237040 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:04:57.237040 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:04:57.237040 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:04:57.237040 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:04:57.237040 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:04:57.237040 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:04:57.237040 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:04:57.237040 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:04:57.237040 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:04:57.237040 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:04:57.237040 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:04:57.237040 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 12:04:57.237040 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 12:04:57.237040 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 12:04:57.237040 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 12:04:57.238427 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:04:57.238427 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:04:57.238427 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 12:04:57.238427 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 12:04:57.238427 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 12:04:57.238427 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 12:04:57.238427 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:04:57.238427 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:04:57.238427 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:04:57.238427 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:04:57.238427 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:04:57.238427 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 25 12:04:57.238427 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:04:57.238427 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:04:57.238427 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:04:57.238427 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:04:57.238427 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:04:57.238427 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:04:57.239715 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:04:57.239715 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:04:57.239715 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 12:04:57.239715 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 12:04:57.239715 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 12:04:57.239715 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 12:04:57.239715 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 12:04:57.239715 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 12:04:57.239715 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 25 12:04:57.239715 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 25 12:04:57.239715 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 617/1239] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jul 25 12:04:57 2022] GET /identity/v3/users/placement => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:04:57.244946 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2730ff58-d592-4f26-979e-b9acb07b81b9 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:04:57.359302 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2730ff58-d592-4f26-979e-b9acb07b81b9 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:04:57.398519 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df42d9e5-df07-4f09-8e3a-3492c632f601 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:04:57.399763 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df42d9e5-df07-4f09-8e3a-3492c632f601 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:04:57.400030 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df42d9e5-df07-4f09-8e3a-3492c632f601 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:04:57.400310 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df42d9e5-df07-4f09-8e3a-3492c632f601 None admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:04:57.401279 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df42d9e5-df07-4f09-8e3a-3492c632f601 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:04:57.401617 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df42d9e5-df07-4f09-8e3a-3492c632f601 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:04:57.403326 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df42d9e5-df07-4f09-8e3a-3492c632f601 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:04:57.424423 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 623/1240] 10.222.0.42 () {56 vars in 1143 bytes} [Mon Jul 25 12:04:57 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:04:57.430319 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc1ba7e-3e4b-4bce-a2c5-e12999c5518b None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:04:57.536630 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc1ba7e-3e4b-4bce-a2c5-e12999c5518b None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:04:57.578645 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2f8ba9-0ee5-44e8-8bf3-2f730640dbe8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:04:57.579392 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2f8ba9-0ee5-44e8-8bf3-2f730640dbe8 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:04:57.579782 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2f8ba9-0ee5-44e8-8bf3-2f730640dbe8 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:04:57.580146 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2f8ba9-0ee5-44e8-8bf3-2f730640dbe8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:04:57.652257 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:04:57.659789 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2f8ba9-0ee5-44e8-8bf3-2f730640dbe8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:04:57.661892 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2f8ba9-0ee5-44e8-8bf3-2f730640dbe8 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:04:57.779021 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 618/1241] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Jul 25 12:04:57 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 25 12:04:57.784763 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df42d9e5-df07-4f09-8e3a-3492c632f601 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:04:57.815838 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:04:57.903802 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df42d9e5-df07-4f09-8e3a-3492c632f601 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:04:57.942760 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb10769-b884-4725-b996-884d4b27d468 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:04:57.943771 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb10769-b884-4725-b996-884d4b27d468 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:04:57.944062 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb10769-b884-4725-b996-884d4b27d468 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:04:57.944346 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb10769-b884-4725-b996-884d4b27d468 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:04:57.955939 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb10769-b884-4725-b996-884d4b27d468 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:04:57.957542 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb10769-b884-4725-b996-884d4b27d468 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:04:57.978278 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-7fb10769-b884-4725-b996-884d4b27d468 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:04:57.978278 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:04:57.978278 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:04:57.978278 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:04:57.978278 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:04:57.978278 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:04:57.978278 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:04:57.978278 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:04:57.978278 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:04:57.978278 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:04:57.978278 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:04:57.978278 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:04:57.978278 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 12:04:57.978278 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 12:04:57.978278 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 12:04:57.979700 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 12:04:57.979700 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:04:57.979700 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:04:57.979700 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:04:57.979700 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:04:57.979700 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:04:57.979700 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:04:57.979700 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:04:57.979700 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jul 25 12:04:57.979700 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:04:57.979700 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:04:57.979700 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:04:57.979700 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:04:57.979700 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:04:57.979700 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:04:57.979700 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:04:57.979700 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:04:57.980848 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 12:04:57.980848 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 12:04:57.980848 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:04:57.980848 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jul 25 12:04:57.980848 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 624/1242] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jul 25 12:04:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:04:57.987584 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2f8ba9-0ee5-44e8-8bf3-2f730640dbe8 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:04:58.092141 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:04:58.098574 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2f8ba9-0ee5-44e8-8bf3-2f730640dbe8 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:04:58.138115 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa3735d-3f8c-4d71-92ff-a0c006171ba9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:04:58.139077 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa3735d-3f8c-4d71-92ff-a0c006171ba9 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:04:58.139412 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa3735d-3f8c-4d71-92ff-a0c006171ba9 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:04:58.139725 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa3735d-3f8c-4d71-92ff-a0c006171ba9 None admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:04:58.140626 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa3735d-3f8c-4d71-92ff-a0c006171ba9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:04:58.140976 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa3735d-3f8c-4d71-92ff-a0c006171ba9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:04:58.142621 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa3735d-3f8c-4d71-92ff-a0c006171ba9 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:04:58.155447 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 619/1243] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jul 25 12:04:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:04:58.162721 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb10769-b884-4725-b996-884d4b27d468 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:04:58.252940 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:04:58.274972 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb10769-b884-4725-b996-884d4b27d468 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:04:58.311499 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5aa2ba-1bf1-4521-a90e-6cd7888e685d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:04:58.312326 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5aa2ba-1bf1-4521-a90e-6cd7888e685d None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:04:58.312605 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5aa2ba-1bf1-4521-a90e-6cd7888e685d None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:04:58.312889 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5aa2ba-1bf1-4521-a90e-6cd7888e685d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:04:58.320040 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5aa2ba-1bf1-4521-a90e-6cd7888e685d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:04:58.321529 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5aa2ba-1bf1-4521-a90e-6cd7888e685d None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:04:58.335359 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-af5aa2ba-1bf1-4521-a90e-6cd7888e685d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 12:04:58.335359 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:04:58.335359 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:04:58.335359 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:04:58.335359 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:04:58.335359 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:04:58.335359 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:04:58.335359 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:04:58.335359 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:04:58.335359 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:04:58.335359 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:04:58.335359 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:04:58.335359 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 12:04:58.335359 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 12:04:58.335359 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 12:04:58.337109 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 12:04:58.337109 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:04:58.337109 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:04:58.337109 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:04:58.337109 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:04:58.337109 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:04:58.337109 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:04:58.337109 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:04:58.337109 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jul 25 12:04:58.337109 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:04:58.337109 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:04:58.337109 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:04:58.337109 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:04:58.337109 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:04:58.337109 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:04:58.337109 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:04:58.337109 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:04:58.338138 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 12:04:58.338138 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 12:04:58.338138 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 12:04:58.338138 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 12:04:58.338138 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 12:04:58.338138 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 12:04:58.338138 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 12:04:58.338138 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jul 25 12:04:58.338138 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 625/1244] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Jul 25 12:04:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:04:58.343342 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa3735d-3f8c-4d71-92ff-a0c006171ba9 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:04:58.453180 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa3735d-3f8c-4d71-92ff-a0c006171ba9 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:04:58.491759 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd57896b-f33f-4590-af2c-aec07b5e2760 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:04:58.492472 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd57896b-f33f-4590-af2c-aec07b5e2760 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:04:58.492760 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd57896b-f33f-4590-af2c-aec07b5e2760 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:04:58.493040 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd57896b-f33f-4590-af2c-aec07b5e2760 None admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:04:58.493825 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd57896b-f33f-4590-af2c-aec07b5e2760 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:04:58.494138 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd57896b-f33f-4590-af2c-aec07b5e2760 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:04:58.495613 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd57896b-f33f-4590-af2c-aec07b5e2760 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:04:58.510059 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 620/1245] 10.222.0.42 () {56 vars in 1148 bytes} [Mon Jul 25 12:04:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:04:58.516351 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5aa2ba-1bf1-4521-a90e-6cd7888e685d None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:04:58.636497 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5aa2ba-1bf1-4521-a90e-6cd7888e685d None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:04:58.674664 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d849b6-46b9-4742-acb9-d3334b4a1171 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:04:58.675433 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d849b6-46b9-4742-acb9-d3334b4a1171 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:04:58.675805 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d849b6-46b9-4742-acb9-d3334b4a1171 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:04:58.676166 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d849b6-46b9-4742-acb9-d3334b4a1171 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:04:58.677136 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d849b6-46b9-4742-acb9-d3334b4a1171 None admin] RBAC: Adding query filter params (role.id=79dec36aee5b4322bf652041c25ca6e4, scope.project.id=b9f62b8f37304b8a9a4bb54205347577, user.id=8012afab6cac4358ab878732e58d0689) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:04:58.677462 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d849b6-46b9-4742-acb9-d3334b4a1171 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:04:58.679056 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d849b6-46b9-4742-acb9-d3334b4a1171 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:04:58.691772 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 626/1246] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Jul 25 12:04:58 2022] GET /identity/v3/role_assignments?user.id=8012afab6cac4358ab878732e58d0689&scope.project.id=b9f62b8f37304b8a9a4bb54205347577&role.id=79dec36aee5b4322bf652041c25ca6e4 => generated 608 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:05:00.943017 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7db58e-6eae-4096-9753-330c78bc37f0 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:05:00.943612 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7db58e-6eae-4096-9753-330c78bc37f0 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:05:00.943737 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7db58e-6eae-4096-9753-330c78bc37f0 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:05:00.945978 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 621/1247] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 25 12:05:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:05:00.951257 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a93a85-c755-4f2f-a80a-835084479705 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:05:00.951723 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a93a85-c755-4f2f-a80a-835084479705 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:05:00.951902 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a93a85-c755-4f2f-a80a-835084479705 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:05:00.959087 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:05:00.973093 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-56a93a85-c755-4f2f-a80a-835084479705 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:05:00.974296 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-56a93a85-c755-4f2f-a80a-835084479705 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:05:01.055771 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 627/1248] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 25 12:05:00 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 12:05:01.082045 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7db58e-6eae-4096-9753-330c78bc37f0 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:05:01.102809 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:05:01.201351 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7db58e-6eae-4096-9753-330c78bc37f0 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:05:01.243000 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f111d4-83bd-4a1a-955e-7fb1c821621d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:05:01.243949 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f111d4-83bd-4a1a-955e-7fb1c821621d None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:05:01.244236 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f111d4-83bd-4a1a-955e-7fb1c821621d None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:05:01.244520 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f111d4-83bd-4a1a-955e-7fb1c821621d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:05:01.255984 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f111d4-83bd-4a1a-955e-7fb1c821621d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 12:05:01.265090 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-a4f111d4-83bd-4a1a-955e-7fb1c821621d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 12:05:01.265090 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:05:01.265090 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:05:01.265090 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:05:01.265090 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:05:01.265090 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:05:01.265090 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:05:01.265090 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:05:01.265090 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:05:01.265090 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:05:01.265090 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:05:01.265090 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:05:01.265090 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 12:05:01.265090 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 12:05:01.265090 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 12:05:01.265090 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 12:05:01.266330 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 12:05:01.266330 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 12:05:01.266330 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 12:05:01.266330 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 12:05:01.266330 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:05:01.266330 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:05:01.266330 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:05:01.266330 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:05:01.266330 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:05:01.266330 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:05:01.266330 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:05:01.266330 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 25 12:05:01.266330 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:05:01.266330 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:05:01.266330 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:05:01.266330 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:05:01.267269 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:05:01.267269 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:05:01.267269 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:05:01.267269 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:05:01.267269 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 12:05:01.267269 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 12:05:01.267269 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 12:05:01.267269 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 12:05:01.267269 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 12:05:01.267269 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 12:05:01.267269 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 12:05:01.267269 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 25 12:05:01.267269 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 622/1249] 10.222.0.42 () {56 vars in 1096 bytes} [Mon Jul 25 12:05:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 12:05:01.273438 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a93a85-c755-4f2f-a80a-835084479705 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:05:01.385246 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a93a85-c755-4f2f-a80a-835084479705 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:05:01.419086 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:05:01.424863 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922a8443-db8d-4d7e-bc00-4981493c721a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:05:01.425542 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922a8443-db8d-4d7e-bc00-4981493c721a None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:05:01.425834 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922a8443-db8d-4d7e-bc00-4981493c721a None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:05:01.426116 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922a8443-db8d-4d7e-bc00-4981493c721a None admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:05:01.426869 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922a8443-db8d-4d7e-bc00-4981493c721a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:05:01.427185 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922a8443-db8d-4d7e-bc00-4981493c721a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:05:01.428596 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922a8443-db8d-4d7e-bc00-4981493c721a None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:05:01.437820 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 628/1250] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Jul 25 12:05:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:05:01.444028 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f111d4-83bd-4a1a-955e-7fb1c821621d None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:05:01.534679 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:05:01.565075 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f111d4-83bd-4a1a-955e-7fb1c821621d None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:05:01.607903 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d8c0d0-6a31-46e4-bf08-792fbe8db684 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:05:01.608775 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d8c0d0-6a31-46e4-bf08-792fbe8db684 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:05:01.609070 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d8c0d0-6a31-46e4-bf08-792fbe8db684 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:05:01.609355 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d8c0d0-6a31-46e4-bf08-792fbe8db684 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:05:01.621633 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d8c0d0-6a31-46e4-bf08-792fbe8db684 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:05:01.623117 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d8c0d0-6a31-46e4-bf08-792fbe8db684 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:05:01.642501 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-84d8c0d0-6a31-46e4-bf08-792fbe8db684 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:05:01.642501 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:05:01.642501 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:05:01.642501 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:05:01.642501 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:05:01.642501 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:05:01.642501 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:05:01.642501 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:05:01.642501 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:05:01.642501 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:05:01.642501 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:05:01.642501 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:05:01.642501 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 12:05:01.642501 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 12:05:01.642501 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 12:05:01.644083 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 12:05:01.644083 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:05:01.644083 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:05:01.644083 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:05:01.644083 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:05:01.644083 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:05:01.644083 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:05:01.644083 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:05:01.644083 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 25 12:05:01.644083 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:05:01.644083 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:05:01.644083 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:05:01.644083 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:05:01.644083 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:05:01.644083 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:05:01.644083 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:05:01.644083 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:05:01.645107 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 12:05:01.645107 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 12:05:01.645107 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:05:01.645107 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 25 12:05:01.645107 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 623/1251] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jul 25 12:05:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:05:01.650022 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922a8443-db8d-4d7e-bc00-4981493c721a None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:05:01.763837 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922a8443-db8d-4d7e-bc00-4981493c721a None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:05:01.804326 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bf075f-bea9-4079-b562-65ec06ceb9f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:05:01.805580 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bf075f-bea9-4079-b562-65ec06ceb9f4 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:05:01.805889 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bf075f-bea9-4079-b562-65ec06ceb9f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:05:01.806181 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bf075f-bea9-4079-b562-65ec06ceb9f4 None admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:05:01.806985 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bf075f-bea9-4079-b562-65ec06ceb9f4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:05:01.807331 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bf075f-bea9-4079-b562-65ec06ceb9f4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:05:01.808900 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bf075f-bea9-4079-b562-65ec06ceb9f4 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:05:01.822200 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 629/1252] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jul 25 12:05:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:05:01.828135 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d8c0d0-6a31-46e4-bf08-792fbe8db684 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:05:01.949318 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d8c0d0-6a31-46e4-bf08-792fbe8db684 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:05:01.982208 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:05:01.991943 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03d43d5-3399-472a-aa2e-6dd1f70edd0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:05:01.993315 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03d43d5-3399-472a-aa2e-6dd1f70edd0a None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:05:01.993665 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03d43d5-3399-472a-aa2e-6dd1f70edd0a None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:05:01.993949 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03d43d5-3399-472a-aa2e-6dd1f70edd0a None admin] PATH_INFO: `/v3/users/placement` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:05:02.002531 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03d43d5-3399-472a-aa2e-6dd1f70edd0a None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:05:02.004024 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03d43d5-3399-472a-aa2e-6dd1f70edd0a None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:05:02.018912 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-e03d43d5-3399-472a-aa2e-6dd1f70edd0a None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 25 12:05:02.018912 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:05:02.018912 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:05:02.018912 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:05:02.018912 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:05:02.018912 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:05:02.018912 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:05:02.018912 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:05:02.018912 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:05:02.018912 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:05:02.018912 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:05:02.018912 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:05:02.018912 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 12:05:02.018912 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 12:05:02.018912 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 12:05:02.018912 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 12:05:02.020098 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:05:02.020098 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:05:02.020098 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 12:05:02.020098 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 12:05:02.020098 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 12:05:02.020098 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 12:05:02.020098 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:05:02.020098 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:05:02.020098 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:05:02.020098 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:05:02.020098 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:05:02.020098 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 25 12:05:02.020098 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:05:02.020098 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:05:02.020098 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:05:02.020098 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:05:02.020098 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:05:02.020098 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:05:02.021230 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:05:02.021230 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:05:02.021230 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 12:05:02.021230 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 12:05:02.021230 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 12:05:02.021230 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 12:05:02.021230 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 12:05:02.021230 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 12:05:02.021230 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 25 12:05:02.021230 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 25 12:05:02.021230 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 624/1253] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jul 25 12:05:01 2022] GET /identity/v3/users/placement => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:05:02.025893 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bf075f-bea9-4079-b562-65ec06ceb9f4 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:05:02.085795 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:05:02.137830 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bf075f-bea9-4079-b562-65ec06ceb9f4 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:05:02.177903 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b34bc2-cb22-4f01-aa17-18412630cd26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:05:02.178670 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b34bc2-cb22-4f01-aa17-18412630cd26 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:05:02.179010 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b34bc2-cb22-4f01-aa17-18412630cd26 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:05:02.179318 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b34bc2-cb22-4f01-aa17-18412630cd26 None admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:05:02.180861 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b34bc2-cb22-4f01-aa17-18412630cd26 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:05:02.181345 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b34bc2-cb22-4f01-aa17-18412630cd26 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:05:02.182821 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b34bc2-cb22-4f01-aa17-18412630cd26 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:05:02.202149 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 630/1254] 10.222.0.42 () {56 vars in 1143 bytes} [Mon Jul 25 12:05:02 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:05:02.207908 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03d43d5-3399-472a-aa2e-6dd1f70edd0a None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:05:02.342537 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03d43d5-3399-472a-aa2e-6dd1f70edd0a None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:05:02.379454 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0f061b-c81a-4589-ab1d-f2bbbc393f8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:05:02.380148 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0f061b-c81a-4589-ab1d-f2bbbc393f8f None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:05:02.380433 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0f061b-c81a-4589-ab1d-f2bbbc393f8f None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:05:02.380717 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0f061b-c81a-4589-ab1d-f2bbbc393f8f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:05:02.444349 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:05:02.459865 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0f061b-c81a-4589-ab1d-f2bbbc393f8f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:05:02.461500 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0f061b-c81a-4589-ab1d-f2bbbc393f8f None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:05:02.568838 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 625/1255] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Jul 25 12:05:02 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 25 12:05:02.575398 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b34bc2-cb22-4f01-aa17-18412630cd26 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:05:02.688949 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b34bc2-cb22-4f01-aa17-18412630cd26 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:05:02.703438 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:05:02.727472 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73439058-f0ed-4812-b736-21ee161240fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:05:02.728217 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73439058-f0ed-4812-b736-21ee161240fe None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:05:02.728500 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73439058-f0ed-4812-b736-21ee161240fe None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:05:02.728777 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73439058-f0ed-4812-b736-21ee161240fe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:05:02.742391 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73439058-f0ed-4812-b736-21ee161240fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:05:02.744237 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73439058-f0ed-4812-b736-21ee161240fe None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:05:02.763032 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-73439058-f0ed-4812-b736-21ee161240fe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:05:02.763032 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:05:02.763032 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:05:02.763032 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:05:02.763032 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:05:02.763032 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:05:02.763032 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:05:02.763032 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:05:02.763032 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:05:02.763032 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:05:02.763032 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:05:02.763032 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:05:02.763032 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 12:05:02.763032 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 12:05:02.763032 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 12:05:02.764168 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 12:05:02.764168 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:05:02.764168 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:05:02.764168 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:05:02.764168 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:05:02.764168 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:05:02.764168 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:05:02.764168 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:05:02.764168 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jul 25 12:05:02.764168 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:05:02.764168 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:05:02.764168 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:05:02.764168 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:05:02.764168 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:05:02.764168 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:05:02.764168 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:05:02.764168 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:05:02.765466 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 12:05:02.765466 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 12:05:02.765466 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:05:02.765466 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jul 25 12:05:02.765466 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 631/1256] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jul 25 12:05:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:05:02.770747 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0f061b-c81a-4589-ab1d-f2bbbc393f8f None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:05:02.867831 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:05:02.880364 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0f061b-c81a-4589-ab1d-f2bbbc393f8f None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:05:02.919594 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5600474f-3806-4da7-b683-019db2bfcd46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:05:02.920259 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5600474f-3806-4da7-b683-019db2bfcd46 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:05:02.920565 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5600474f-3806-4da7-b683-019db2bfcd46 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:05:02.920842 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5600474f-3806-4da7-b683-019db2bfcd46 None admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:05:02.921559 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5600474f-3806-4da7-b683-019db2bfcd46 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:05:02.922660 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5600474f-3806-4da7-b683-019db2bfcd46 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:05:02.923890 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5600474f-3806-4da7-b683-019db2bfcd46 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:05:02.936054 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 626/1257] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jul 25 12:05:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:05:02.942179 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73439058-f0ed-4812-b736-21ee161240fe None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:05:03.050412 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73439058-f0ed-4812-b736-21ee161240fe None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:05:03.087008 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f4a3df-1c80-4604-b80e-68dea0efcb10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:05:03.087997 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f4a3df-1c80-4604-b80e-68dea0efcb10 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:05:03.088288 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f4a3df-1c80-4604-b80e-68dea0efcb10 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:05:03.088625 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f4a3df-1c80-4604-b80e-68dea0efcb10 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:05:03.095505 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f4a3df-1c80-4604-b80e-68dea0efcb10 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:05:03.097036 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f4a3df-1c80-4604-b80e-68dea0efcb10 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:05:03.113139 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-67f4a3df-1c80-4604-b80e-68dea0efcb10 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 12:05:03.113139 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:05:03.113139 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:05:03.113139 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:05:03.113139 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:05:03.113139 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:05:03.113139 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:05:03.113139 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:05:03.113139 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:05:03.113139 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:05:03.113139 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:05:03.113139 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:05:03.113139 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 12:05:03.113139 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 12:05:03.113139 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 12:05:03.114281 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 12:05:03.114281 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:05:03.114281 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:05:03.114281 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:05:03.114281 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:05:03.114281 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:05:03.114281 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:05:03.114281 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:05:03.114281 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jul 25 12:05:03.114281 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:05:03.114281 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:05:03.114281 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:05:03.114281 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:05:03.114281 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:05:03.114281 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:05:03.114281 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:05:03.114281 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:05:03.115217 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 12:05:03.115217 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 12:05:03.115217 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 12:05:03.115217 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 12:05:03.115217 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 12:05:03.115217 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 12:05:03.115217 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 12:05:03.115217 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jul 25 12:05:03.115217 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 632/1258] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Jul 25 12:05:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:05:03.121100 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5600474f-3806-4da7-b683-019db2bfcd46 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:05:03.239820 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5600474f-3806-4da7-b683-019db2bfcd46 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:05:03.276289 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c030fee1-7f4c-4ff7-8ae4-042419b9d3be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:05:03.276999 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c030fee1-7f4c-4ff7-8ae4-042419b9d3be None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:05:03.277257 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c030fee1-7f4c-4ff7-8ae4-042419b9d3be None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:05:03.277542 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c030fee1-7f4c-4ff7-8ae4-042419b9d3be None admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:05:03.278320 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c030fee1-7f4c-4ff7-8ae4-042419b9d3be None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:05:03.278638 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c030fee1-7f4c-4ff7-8ae4-042419b9d3be None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:05:03.280114 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c030fee1-7f4c-4ff7-8ae4-042419b9d3be None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:05:03.294265 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 627/1259] 10.222.0.42 () {56 vars in 1148 bytes} [Mon Jul 25 12:05:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:05:03.300343 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f4a3df-1c80-4604-b80e-68dea0efcb10 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:05:03.330212 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:05:03.426290 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f4a3df-1c80-4604-b80e-68dea0efcb10 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:05:03.462340 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387ee814-4cff-4342-95e3-078c114bab38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:05:03.463014 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387ee814-4cff-4342-95e3-078c114bab38 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:05:03.463308 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387ee814-4cff-4342-95e3-078c114bab38 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:05:03.463625 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387ee814-4cff-4342-95e3-078c114bab38 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:05:03.464473 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387ee814-4cff-4342-95e3-078c114bab38 None admin] RBAC: Adding query filter params (role.id=a7d20885b6d94928a34b3cc5b3a00006, scope.project.id=b9f62b8f37304b8a9a4bb54205347577, user.id=8012afab6cac4358ab878732e58d0689) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:05:03.464796 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387ee814-4cff-4342-95e3-078c114bab38 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:05:03.466243 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387ee814-4cff-4342-95e3-078c114bab38 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:05:03.472256 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 633/1260] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Jul 25 12:05:03 2022] GET /identity/v3/role_assignments?user.id=8012afab6cac4358ab878732e58d0689&scope.project.id=b9f62b8f37304b8a9a4bb54205347577&role.id=a7d20885b6d94928a34b3cc5b3a00006 => generated 259 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:05:05.469387 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c364706-fab9-419f-8583-5383d9c61c29 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:05:05.469922 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c364706-fab9-419f-8583-5383d9c61c29 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:05:05.470015 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c364706-fab9-419f-8583-5383d9c61c29 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:05:05.471311 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 628/1261] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 25 12:05:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:05:05.476373 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8c4b36-e4fb-4810-9865-e3431ede6a47 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:05:05.476744 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8c4b36-e4fb-4810-9865-e3431ede6a47 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:05:05.476931 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8c4b36-e4fb-4810-9865-e3431ede6a47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:05:05.494562 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-bf8c4b36-e4fb-4810-9865-e3431ede6a47 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:05:05.495775 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-bf8c4b36-e4fb-4810-9865-e3431ede6a47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:05:05.567587 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 634/1262] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 25 12:05:05 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 12:05:05.592572 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c364706-fab9-419f-8583-5383d9c61c29 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:05:05.604037 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:05:05.693392 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c364706-fab9-419f-8583-5383d9c61c29 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:05:05.725883 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9879b09a-c58b-40cd-a3c8-bfd59a5cd606 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:05:05.726623 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9879b09a-c58b-40cd-a3c8-bfd59a5cd606 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:05:05.726907 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9879b09a-c58b-40cd-a3c8-bfd59a5cd606 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:05:05.727196 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9879b09a-c58b-40cd-a3c8-bfd59a5cd606 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:05:05.736910 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9879b09a-c58b-40cd-a3c8-bfd59a5cd606 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 12:05:05.744946 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-9879b09a-c58b-40cd-a3c8-bfd59a5cd606 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 12:05:05.744946 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:05:05.744946 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:05:05.744946 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:05:05.744946 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:05:05.744946 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:05:05.744946 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:05:05.744946 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:05:05.744946 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:05:05.744946 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:05:05.744946 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:05:05.744946 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:05:05.744946 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 12:05:05.744946 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 12:05:05.744946 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 12:05:05.744946 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 12:05:05.746055 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 12:05:05.746055 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 12:05:05.746055 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 12:05:05.746055 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 12:05:05.746055 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:05:05.746055 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:05:05.746055 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:05:05.746055 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:05:05.746055 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:05:05.746055 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:05:05.746055 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:05:05.746055 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 25 12:05:05.746055 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:05:05.746055 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:05:05.746055 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:05:05.746055 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:05:05.747139 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:05:05.747139 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:05:05.747139 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:05:05.747139 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:05:05.747139 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 12:05:05.747139 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 12:05:05.747139 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 12:05:05.747139 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 12:05:05.747139 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 12:05:05.747139 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 12:05:05.747139 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 12:05:05.747139 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 25 12:05:05.747139 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 629/1263] 10.222.0.42 () {56 vars in 1096 bytes} [Mon Jul 25 12:05:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 12:05:05.752101 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8c4b36-e4fb-4810-9865-e3431ede6a47 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:05:05.770380 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:05:05.849414 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8c4b36-e4fb-4810-9865-e3431ede6a47 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:05:05.883498 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f23d3f-9235-4d1a-969f-aba9bf6508b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:05:05.884188 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f23d3f-9235-4d1a-969f-aba9bf6508b6 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:05:05.884474 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f23d3f-9235-4d1a-969f-aba9bf6508b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:05:05.884750 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f23d3f-9235-4d1a-969f-aba9bf6508b6 None admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:05:05.885490 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f23d3f-9235-4d1a-969f-aba9bf6508b6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:05:05.885799 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f23d3f-9235-4d1a-969f-aba9bf6508b6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:05:05.887172 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f23d3f-9235-4d1a-969f-aba9bf6508b6 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:05:05.894991 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 635/1264] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Jul 25 12:05:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:05:05.900786 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9879b09a-c58b-40cd-a3c8-bfd59a5cd606 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:05:05.965331 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:05:05.998637 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9879b09a-c58b-40cd-a3c8-bfd59a5cd606 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:05:06.030178 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76dc4f88-54f3-413d-8df1-4def21d6e9c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:05:06.030947 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dc4f88-54f3-413d-8df1-4def21d6e9c3 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:05:06.031232 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dc4f88-54f3-413d-8df1-4def21d6e9c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:05:06.031509 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dc4f88-54f3-413d-8df1-4def21d6e9c3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:05:06.042419 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76dc4f88-54f3-413d-8df1-4def21d6e9c3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:05:06.043888 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76dc4f88-54f3-413d-8df1-4def21d6e9c3 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:05:06.060839 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-76dc4f88-54f3-413d-8df1-4def21d6e9c3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:05:06.060839 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:05:06.060839 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:05:06.060839 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:05:06.060839 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:05:06.060839 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:05:06.060839 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:05:06.060839 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:05:06.060839 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:05:06.060839 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:05:06.060839 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:05:06.060839 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:05:06.060839 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 12:05:06.060839 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 12:05:06.060839 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 12:05:06.061877 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 12:05:06.061877 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:05:06.061877 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:05:06.061877 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:05:06.061877 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:05:06.061877 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:05:06.061877 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:05:06.061877 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:05:06.061877 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 25 12:05:06.061877 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:05:06.061877 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:05:06.061877 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:05:06.061877 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:05:06.061877 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:05:06.061877 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:05:06.061877 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:05:06.061877 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:05:06.062987 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 12:05:06.062987 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 12:05:06.062987 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:05:06.062987 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 25 12:05:06.062987 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 630/1265] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jul 25 12:05:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:05:06.067690 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f23d3f-9235-4d1a-969f-aba9bf6508b6 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:05:06.146057 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:05:06.165511 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f23d3f-9235-4d1a-969f-aba9bf6508b6 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:05:06.199208 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33ab52b-5e7a-4ee8-b691-ac76c48a9f56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:05:06.199881 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33ab52b-5e7a-4ee8-b691-ac76c48a9f56 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:05:06.200173 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33ab52b-5e7a-4ee8-b691-ac76c48a9f56 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:05:06.200453 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33ab52b-5e7a-4ee8-b691-ac76c48a9f56 None admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:05:06.201167 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33ab52b-5e7a-4ee8-b691-ac76c48a9f56 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:05:06.201477 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33ab52b-5e7a-4ee8-b691-ac76c48a9f56 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:05:06.202842 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33ab52b-5e7a-4ee8-b691-ac76c48a9f56 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:05:06.214234 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 636/1266] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jul 25 12:05:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 12:05:06.219729 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76dc4f88-54f3-413d-8df1-4def21d6e9c3 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:05:06.317228 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76dc4f88-54f3-413d-8df1-4def21d6e9c3 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:05:06.336377 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:05:06.349990 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf22dfd-72eb-4275-a667-d883e3112884 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:05:06.350772 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf22dfd-72eb-4275-a667-d883e3112884 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:05:06.351684 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf22dfd-72eb-4275-a667-d883e3112884 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:05:06.351976 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf22dfd-72eb-4275-a667-d883e3112884 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:05:06.359131 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf22dfd-72eb-4275-a667-d883e3112884 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:05:06.360608 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf22dfd-72eb-4275-a667-d883e3112884 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:05:06.374482 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-faf22dfd-72eb-4275-a667-d883e3112884 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 25 12:05:06.374482 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:05:06.374482 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:05:06.374482 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:05:06.374482 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:05:06.374482 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:05:06.374482 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:05:06.374482 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:05:06.374482 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:05:06.374482 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:05:06.374482 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:05:06.374482 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:05:06.374482 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 12:05:06.374482 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 12:05:06.374482 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 12:05:06.374482 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 12:05:06.375609 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:05:06.375609 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:05:06.375609 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 12:05:06.375609 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 12:05:06.375609 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 12:05:06.375609 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 12:05:06.375609 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:05:06.375609 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:05:06.375609 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:05:06.375609 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:05:06.375609 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:05:06.375609 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 25 12:05:06.375609 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:05:06.375609 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:05:06.375609 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:05:06.375609 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:05:06.375609 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:05:06.375609 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:05:06.376599 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:05:06.376599 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:05:06.376599 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 12:05:06.376599 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 12:05:06.376599 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 12:05:06.376599 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 12:05:06.376599 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 12:05:06.376599 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 12:05:06.376599 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 25 12:05:06.376599 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 25 12:05:06.376599 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 631/1267] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jul 25 12:05:06 2022] GET /identity/v3/users/placement => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:05:06.381165 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33ab52b-5e7a-4ee8-b691-ac76c48a9f56 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:05:06.478059 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33ab52b-5e7a-4ee8-b691-ac76c48a9f56 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:05:06.510190 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355946af-ac5c-4063-8e95-3d8f64f9a15a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:05:06.510855 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355946af-ac5c-4063-8e95-3d8f64f9a15a None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:05:06.511141 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355946af-ac5c-4063-8e95-3d8f64f9a15a None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:05:06.511420 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355946af-ac5c-4063-8e95-3d8f64f9a15a None admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:05:06.512277 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355946af-ac5c-4063-8e95-3d8f64f9a15a None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:05:06.512594 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355946af-ac5c-4063-8e95-3d8f64f9a15a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:05:06.514005 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355946af-ac5c-4063-8e95-3d8f64f9a15a None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:05:06.522374 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:05:06.532095 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 637/1268] 10.222.0.42 () {56 vars in 1143 bytes} [Mon Jul 25 12:05:06 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:05:06.538142 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf22dfd-72eb-4275-a667-d883e3112884 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:05:06.642673 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf22dfd-72eb-4275-a667-d883e3112884 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:05:06.675951 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fbcc23-0018-4bdf-b6bf-225f58334703 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:05:06.676734 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fbcc23-0018-4bdf-b6bf-225f58334703 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:05:06.677024 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fbcc23-0018-4bdf-b6bf-225f58334703 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:05:06.677311 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fbcc23-0018-4bdf-b6bf-225f58334703 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:05:06.687560 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64fbcc23-0018-4bdf-b6bf-225f58334703 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:05:06.689015 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64fbcc23-0018-4bdf-b6bf-225f58334703 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:05:06.705720 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-64fbcc23-0018-4bdf-b6bf-225f58334703 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:05:06.705720 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:05:06.705720 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:05:06.705720 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:05:06.705720 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:05:06.705720 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:05:06.705720 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:05:06.705720 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:05:06.705720 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:05:06.705720 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:05:06.705720 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:05:06.705720 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:05:06.705720 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 12:05:06.705720 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 12:05:06.705720 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 12:05:06.706762 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 12:05:06.706762 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:05:06.706762 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:05:06.706762 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:05:06.706762 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:05:06.706762 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:05:06.706762 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:05:06.706762 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:05:06.706762 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 25 12:05:06.706762 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:05:06.706762 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:05:06.706762 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:05:06.706762 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:05:06.706762 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:05:06.706762 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:05:06.706762 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:05:06.706762 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:05:06.707913 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 12:05:06.707913 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 12:05:06.707913 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:05:06.707913 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 25 12:05:06.707913 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 632/1269] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jul 25 12:05:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:05:06.713249 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355946af-ac5c-4063-8e95-3d8f64f9a15a None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:05:06.815168 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355946af-ac5c-4063-8e95-3d8f64f9a15a None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:05:06.849215 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fc9daf-551b-4ceb-83a5-28bc9775f5cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:05:06.849865 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fc9daf-551b-4ceb-83a5-28bc9775f5cc None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:05:06.850148 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fc9daf-551b-4ceb-83a5-28bc9775f5cc None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:05:06.850425 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fc9daf-551b-4ceb-83a5-28bc9775f5cc None admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:05:06.851135 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fc9daf-551b-4ceb-83a5-28bc9775f5cc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:05:06.851440 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fc9daf-551b-4ceb-83a5-28bc9775f5cc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:05:06.852822 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fc9daf-551b-4ceb-83a5-28bc9775f5cc None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:05:06.863746 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 638/1270] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jul 25 12:05:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:05:06.869671 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fbcc23-0018-4bdf-b6bf-225f58334703 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:05:06.901177 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:05:06.967159 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fbcc23-0018-4bdf-b6bf-225f58334703 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:05:06.999721 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0004c6-0205-4434-bd4a-925740e71fdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:05:07.000495 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0004c6-0205-4434-bd4a-925740e71fdb None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:05:07.000786 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0004c6-0205-4434-bd4a-925740e71fdb None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:05:07.001070 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0004c6-0205-4434-bd4a-925740e71fdb None admin] PATH_INFO: `/v3/projects/service` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:05:07.007502 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0004c6-0205-4434-bd4a-925740e71fdb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:05:07.008971 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0004c6-0205-4434-bd4a-925740e71fdb None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:05:07.022746 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-cf0004c6-0205-4434-bd4a-925740e71fdb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 12:05:07.022746 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:05:07.022746 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:05:07.022746 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:05:07.022746 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:05:07.022746 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:05:07.022746 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:05:07.022746 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:05:07.022746 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:05:07.022746 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:05:07.022746 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:05:07.022746 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:05:07.022746 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 12:05:07.022746 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 12:05:07.022746 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 12:05:07.023805 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 12:05:07.023805 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:05:07.023805 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:05:07.023805 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:05:07.023805 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:05:07.023805 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:05:07.023805 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:05:07.023805 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:05:07.023805 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 25 12:05:07.023805 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:05:07.023805 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:05:07.023805 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:05:07.023805 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:05:07.023805 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:05:07.023805 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:05:07.023805 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:05:07.023805 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:05:07.024920 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 12:05:07.024920 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 12:05:07.024920 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 12:05:07.024920 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 12:05:07.024920 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 12:05:07.024920 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 12:05:07.024920 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 12:05:07.024920 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 25 12:05:07.024920 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 633/1271] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Jul 25 12:05:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:05:07.029681 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fc9daf-551b-4ceb-83a5-28bc9775f5cc None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:05:07.086255 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:05:07.126456 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fc9daf-551b-4ceb-83a5-28bc9775f5cc None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:05:07.158551 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fc1d10-71f1-434e-bf10-7a6c5c9e70ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:05:07.159140 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fc1d10-71f1-434e-bf10-7a6c5c9e70ae None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:05:07.159423 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fc1d10-71f1-434e-bf10-7a6c5c9e70ae None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:05:07.159729 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fc1d10-71f1-434e-bf10-7a6c5c9e70ae None admin] PATH_INFO: `/v3/projects` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:05:07.160508 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fc1d10-71f1-434e-bf10-7a6c5c9e70ae None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:05:07.160818 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fc1d10-71f1-434e-bf10-7a6c5c9e70ae None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:05:07.162506 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fc1d10-71f1-434e-bf10-7a6c5c9e70ae None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:05:07.174554 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 639/1272] 10.222.0.42 () {56 vars in 1148 bytes} [Mon Jul 25 12:05:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:05:07.180563 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0004c6-0205-4434-bd4a-925740e71fdb None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:05:07.268491 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:05:07.277602 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0004c6-0205-4434-bd4a-925740e71fdb None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:05:07.310381 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51456488-7bea-425c-982a-09597665de09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:05:07.311253 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51456488-7bea-425c-982a-09597665de09 None admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:05:07.311562 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51456488-7bea-425c-982a-09597665de09 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:05:07.311850 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51456488-7bea-425c-982a-09597665de09 None admin] PATH_INFO: `/v3/projects/b9f62b8f37304b8a9a4bb54205347577/users/8012afab6cac4358ab878732e58d0689/roles/a7d20885b6d94928a34b3cc5b3a00006` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:05:07.313785 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51456488-7bea-425c-982a-09597665de09 None admin] RBAC: Authorizing `identity:create_grant(project_id=b9f62b8f37304b8a9a4bb54205347577, user_id=8012afab6cac4358ab878732e58d0689, role_id=a7d20885b6d94928a34b3cc5b3a00006)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:05:07.315209 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51456488-7bea-425c-982a-09597665de09 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:05:07.328069 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 634/1273] 10.222.0.42 () {58 vars in 1439 bytes} [Mon Jul 25 12:05:07 2022] PUT /identity/v3/projects/b9f62b8f37304b8a9a4bb54205347577/users/8012afab6cac4358ab878732e58d0689/roles/a7d20885b6d94928a34b3cc5b3a00006 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 12:05:09.261976 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3feba1b-294b-4796-9a39-7c298e97acee None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:05:09.262502 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3feba1b-294b-4796-9a39-7c298e97acee None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:05:09.262624 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3feba1b-294b-4796-9a39-7c298e97acee None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:05:09.263918 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 640/1274] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 25 12:05:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:05:09.268917 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb86fe9-ea4d-4e84-bf98-e8e2681e2e46 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:05:09.269350 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb86fe9-ea4d-4e84-bf98-e8e2681e2e46 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:05:09.269479 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb86fe9-ea4d-4e84-bf98-e8e2681e2e46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:05:09.287819 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-ecb86fe9-ea4d-4e84-bf98-e8e2681e2e46 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:05:09.289516 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-ecb86fe9-ea4d-4e84-bf98-e8e2681e2e46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:05:09.355399 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 635/1275] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 25 12:05:09 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 12:05:09.377908 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3feba1b-294b-4796-9a39-7c298e97acee None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:05:09.477145 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3feba1b-294b-4796-9a39-7c298e97acee None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:05:09.493441 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:05:09.509637 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610097fe-dd2a-4fbe-b6a5-a42efb81c981 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:05:09.510410 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610097fe-dd2a-4fbe-b6a5-a42efb81c981 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:05:09.510694 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610097fe-dd2a-4fbe-b6a5-a42efb81c981 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:05:09.510970 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610097fe-dd2a-4fbe-b6a5-a42efb81c981 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:05:09.520698 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610097fe-dd2a-4fbe-b6a5-a42efb81c981 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 12:05:09.528551 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-610097fe-dd2a-4fbe-b6a5-a42efb81c981 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 12:05:09.528551 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:05:09.528551 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:05:09.528551 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:05:09.528551 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:05:09.528551 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:05:09.528551 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:05:09.528551 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:05:09.528551 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:05:09.528551 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:05:09.528551 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:05:09.528551 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:05:09.528551 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 12:05:09.528551 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 12:05:09.528551 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 12:05:09.528551 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 12:05:09.529623 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 12:05:09.529623 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 12:05:09.529623 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 12:05:09.529623 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 12:05:09.529623 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:05:09.529623 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:05:09.529623 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:05:09.529623 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:05:09.529623 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:05:09.529623 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:05:09.529623 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:05:09.529623 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jul 25 12:05:09.529623 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:05:09.529623 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:05:09.529623 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:05:09.529623 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:05:09.530518 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:05:09.530518 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:05:09.530518 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:05:09.530518 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:05:09.530518 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 12:05:09.530518 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 12:05:09.530518 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 12:05:09.530518 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 12:05:09.530518 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 12:05:09.530518 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 12:05:09.530518 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 12:05:09.530518 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jul 25 12:05:09.530518 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 641/1276] 10.222.0.42 () {56 vars in 1096 bytes} [Mon Jul 25 12:05:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:05:09.536827 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb86fe9-ea4d-4e84-bf98-e8e2681e2e46 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:05:09.619314 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:05:09.637848 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb86fe9-ea4d-4e84-bf98-e8e2681e2e46 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:05:09.671446 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38ff914-7859-4af5-a8b7-b89316c7a460 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:05:09.672138 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38ff914-7859-4af5-a8b7-b89316c7a460 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:05:09.672429 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38ff914-7859-4af5-a8b7-b89316c7a460 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:05:09.672739 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38ff914-7859-4af5-a8b7-b89316c7a460 None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:05:09.673487 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38ff914-7859-4af5-a8b7-b89316c7a460 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:05:09.673801 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38ff914-7859-4af5-a8b7-b89316c7a460 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:05:09.675190 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38ff914-7859-4af5-a8b7-b89316c7a460 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:05:09.683282 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 636/1277] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Jul 25 12:05:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:05:09.688935 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610097fe-dd2a-4fbe-b6a5-a42efb81c981 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:05:09.787246 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610097fe-dd2a-4fbe-b6a5-a42efb81c981 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:05:09.819263 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed204b9b-6665-4017-91aa-696507cc0148 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:05:09.820590 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed204b9b-6665-4017-91aa-696507cc0148 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:05:09.820873 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed204b9b-6665-4017-91aa-696507cc0148 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:05:09.821150 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed204b9b-6665-4017-91aa-696507cc0148 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:05:09.831563 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed204b9b-6665-4017-91aa-696507cc0148 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:05:09.833057 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed204b9b-6665-4017-91aa-696507cc0148 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:05:09.850420 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-ed204b9b-6665-4017-91aa-696507cc0148 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:05:09.850420 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:05:09.850420 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:05:09.850420 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:05:09.850420 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:05:09.850420 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:05:09.850420 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:05:09.850420 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:05:09.850420 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:05:09.850420 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:05:09.850420 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:05:09.850420 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:05:09.850420 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 12:05:09.850420 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 12:05:09.850420 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 12:05:09.851438 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 12:05:09.851438 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:05:09.851438 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:05:09.851438 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:05:09.851438 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:05:09.851438 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:05:09.851438 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:05:09.851438 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:05:09.851438 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jul 25 12:05:09.851438 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:05:09.851438 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:05:09.851438 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:05:09.851438 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:05:09.851438 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:05:09.851438 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:05:09.851438 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:05:09.851438 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:05:09.852538 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 12:05:09.852538 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 12:05:09.852538 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:05:09.852538 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jul 25 12:05:09.852538 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 642/1278] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jul 25 12:05:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:05:09.857292 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38ff914-7859-4af5-a8b7-b89316c7a460 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:05:09.953960 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38ff914-7859-4af5-a8b7-b89316c7a460 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:05:09.986428 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a14be7-a573-41b3-8e1a-aed21d090220 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:05:09.987081 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a14be7-a573-41b3-8e1a-aed21d090220 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:05:09.987372 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a14be7-a573-41b3-8e1a-aed21d090220 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:05:09.987693 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a14be7-a573-41b3-8e1a-aed21d090220 None admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:05:09.988410 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a14be7-a573-41b3-8e1a-aed21d090220 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:05:09.988720 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a14be7-a573-41b3-8e1a-aed21d090220 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:05:09.990075 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a14be7-a573-41b3-8e1a-aed21d090220 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:05:10.001421 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 637/1279] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jul 25 12:05:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:05:10.007663 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed204b9b-6665-4017-91aa-696507cc0148 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:05:10.039264 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:05:10.106855 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed204b9b-6665-4017-91aa-696507cc0148 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:05:10.138627 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6090a37-82e5-4e02-9554-b809fa24585f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:05:10.139410 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6090a37-82e5-4e02-9554-b809fa24585f None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:05:10.139724 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6090a37-82e5-4e02-9554-b809fa24585f None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:05:10.140004 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6090a37-82e5-4e02-9554-b809fa24585f None admin] PATH_INFO: `/v3/users/placement` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:05:10.147170 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6090a37-82e5-4e02-9554-b809fa24585f None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:05:10.149112 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6090a37-82e5-4e02-9554-b809fa24585f None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:05:10.163121 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-d6090a37-82e5-4e02-9554-b809fa24585f None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 25 12:05:10.163121 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:05:10.163121 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:05:10.163121 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:05:10.163121 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:05:10.163121 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:05:10.163121 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:05:10.163121 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:05:10.163121 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:05:10.163121 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:05:10.163121 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:05:10.163121 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:05:10.163121 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 12:05:10.163121 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 12:05:10.163121 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 12:05:10.163121 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 12:05:10.164742 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:05:10.164742 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:05:10.164742 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 12:05:10.164742 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 12:05:10.164742 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 12:05:10.164742 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 12:05:10.164742 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:05:10.164742 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:05:10.164742 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:05:10.164742 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:05:10.164742 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:05:10.164742 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jul 25 12:05:10.164742 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:05:10.164742 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:05:10.164742 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:05:10.164742 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:05:10.164742 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:05:10.164742 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:05:10.165918 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:05:10.165918 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:05:10.165918 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 12:05:10.165918 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 12:05:10.165918 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 12:05:10.165918 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 12:05:10.165918 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 12:05:10.165918 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 12:05:10.165918 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 25 12:05:10.165918 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jul 25 12:05:10.165918 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 643/1280] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jul 25 12:05:10 2022] GET /identity/v3/users/placement => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:05:10.170473 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a14be7-a573-41b3-8e1a-aed21d090220 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:05:10.174214 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:05:10.267450 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a14be7-a573-41b3-8e1a-aed21d090220 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:05:10.302225 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e4ad90-5919-426b-be69-1e51db7ae2d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:05:10.302886 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e4ad90-5919-426b-be69-1e51db7ae2d9 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:05:10.303172 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e4ad90-5919-426b-be69-1e51db7ae2d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:05:10.303451 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e4ad90-5919-426b-be69-1e51db7ae2d9 None admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:05:10.304861 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e4ad90-5919-426b-be69-1e51db7ae2d9 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:05:10.305201 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e4ad90-5919-426b-be69-1e51db7ae2d9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:05:10.306623 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e4ad90-5919-426b-be69-1e51db7ae2d9 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:05:10.334195 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 638/1281] 10.222.0.42 () {56 vars in 1143 bytes} [Mon Jul 25 12:05:10 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:05:10.340479 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6090a37-82e5-4e02-9554-b809fa24585f None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:05:10.430139 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:05:10.440425 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6090a37-82e5-4e02-9554-b809fa24585f None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:05:10.473080 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28697f43-d2c4-474c-9580-28a935feacb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:05:10.473719 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28697f43-d2c4-474c-9580-28a935feacb4 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:05:10.474009 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28697f43-d2c4-474c-9580-28a935feacb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:05:10.474291 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28697f43-d2c4-474c-9580-28a935feacb4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:05:10.542390 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28697f43-d2c4-474c-9580-28a935feacb4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:05:10.544258 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28697f43-d2c4-474c-9580-28a935feacb4 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:05:10.633556 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:05:10.643978 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 644/1282] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Jul 25 12:05:10 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 25 12:05:10.650558 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e4ad90-5919-426b-be69-1e51db7ae2d9 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:05:10.713621 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:05:10.750269 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e4ad90-5919-426b-be69-1e51db7ae2d9 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:05:10.783571 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d69fb5-a099-4f82-9304-cafad4040148 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:05:10.784365 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d69fb5-a099-4f82-9304-cafad4040148 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:05:10.784647 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d69fb5-a099-4f82-9304-cafad4040148 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:05:10.784926 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d69fb5-a099-4f82-9304-cafad4040148 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:05:10.794981 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d69fb5-a099-4f82-9304-cafad4040148 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:05:10.796438 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d69fb5-a099-4f82-9304-cafad4040148 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:05:10.813337 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-a9d69fb5-a099-4f82-9304-cafad4040148 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:05:10.813337 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:05:10.813337 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:05:10.813337 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:05:10.813337 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:05:10.813337 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:05:10.813337 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:05:10.813337 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:05:10.813337 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:05:10.813337 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:05:10.813337 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:05:10.813337 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:05:10.813337 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 12:05:10.813337 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 12:05:10.813337 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 12:05:10.814355 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 12:05:10.814355 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:05:10.814355 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:05:10.814355 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:05:10.814355 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:05:10.814355 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:05:10.814355 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:05:10.814355 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:05:10.814355 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 25 12:05:10.814355 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:05:10.814355 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:05:10.814355 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:05:10.814355 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:05:10.814355 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:05:10.814355 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:05:10.814355 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:05:10.814355 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:05:10.815384 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 12:05:10.815384 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 12:05:10.815384 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 12:05:10.815384 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 25 12:05:10.815384 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 639/1283] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jul 25 12:05:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:05:10.822709 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28697f43-d2c4-474c-9580-28a935feacb4 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:05:10.924094 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28697f43-d2c4-474c-9580-28a935feacb4 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:05:10.957522 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2113df-56f5-4b45-973c-da067e32b5df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:05:10.958172 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2113df-56f5-4b45-973c-da067e32b5df None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:05:10.958462 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2113df-56f5-4b45-973c-da067e32b5df None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:05:10.958748 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2113df-56f5-4b45-973c-da067e32b5df None admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:05:10.959466 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2113df-56f5-4b45-973c-da067e32b5df None admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:05:10.959811 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2113df-56f5-4b45-973c-da067e32b5df None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:05:10.961207 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2113df-56f5-4b45-973c-da067e32b5df None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:05:10.973616 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 645/1284] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Jul 25 12:05:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:05:10.979260 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d69fb5-a099-4f82-9304-cafad4040148 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:05:11.076154 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d69fb5-a099-4f82-9304-cafad4040148 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:05:11.089407 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:05:11.108564 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d050b913-2435-4a60-bb55-db9f5ec350dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:05:11.109340 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d050b913-2435-4a60-bb55-db9f5ec350dc None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:05:11.109622 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d050b913-2435-4a60-bb55-db9f5ec350dc None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:05:11.109900 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d050b913-2435-4a60-bb55-db9f5ec350dc None admin] PATH_INFO: `/v3/projects/service` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:05:11.116811 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d050b913-2435-4a60-bb55-db9f5ec350dc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:05:11.118257 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d050b913-2435-4a60-bb55-db9f5ec350dc None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:05:11.131445 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-d050b913-2435-4a60-bb55-db9f5ec350dc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 12:05:11.131445 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:05:11.131445 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:05:11.131445 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:05:11.131445 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:05:11.131445 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:05:11.131445 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:05:11.131445 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:05:11.131445 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:05:11.131445 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:05:11.131445 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:05:11.131445 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:05:11.131445 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 12:05:11.131445 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 12:05:11.131445 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 12:05:11.133019 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 12:05:11.133019 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:05:11.133019 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:05:11.133019 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:05:11.133019 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:05:11.133019 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:05:11.133019 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:05:11.133019 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:05:11.133019 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jul 25 12:05:11.133019 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:05:11.133019 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:05:11.133019 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:05:11.133019 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:05:11.133019 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:05:11.133019 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:05:11.133019 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:05:11.133019 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:05:11.134117 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 12:05:11.134117 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 12:05:11.134117 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 12:05:11.134117 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 12:05:11.134117 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 12:05:11.134117 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 12:05:11.134117 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 12:05:11.134117 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jul 25 12:05:11.134117 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 640/1285] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Jul 25 12:05:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:05:11.138990 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2113df-56f5-4b45-973c-da067e32b5df None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:05:11.188310 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:05:11.238622 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2113df-56f5-4b45-973c-da067e32b5df None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:05:11.271292 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5207b0-171c-4e4c-acee-0af292a551ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:05:11.271969 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5207b0-171c-4e4c-acee-0af292a551ed None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:05:11.272346 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5207b0-171c-4e4c-acee-0af292a551ed None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:05:11.272666 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5207b0-171c-4e4c-acee-0af292a551ed None admin] PATH_INFO: `/v3/projects` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:05:11.273449 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5207b0-171c-4e4c-acee-0af292a551ed None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:05:11.273766 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5207b0-171c-4e4c-acee-0af292a551ed None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:05:11.275190 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5207b0-171c-4e4c-acee-0af292a551ed None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:05:11.287059 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 646/1286] 10.222.0.42 () {56 vars in 1148 bytes} [Mon Jul 25 12:05:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:05:11.293443 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d050b913-2435-4a60-bb55-db9f5ec350dc None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:05:11.388850 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d050b913-2435-4a60-bb55-db9f5ec350dc None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:05:11.421135 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b909c3d7-899f-4020-bb00-eddaa27509ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:05:11.421769 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b909c3d7-899f-4020-bb00-eddaa27509ec None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:05:11.422051 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b909c3d7-899f-4020-bb00-eddaa27509ec None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:05:11.422326 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b909c3d7-899f-4020-bb00-eddaa27509ec None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:05:11.423140 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b909c3d7-899f-4020-bb00-eddaa27509ec None admin] RBAC: Adding query filter params (role.id=a7d20885b6d94928a34b3cc5b3a00006, scope.project.id=b9f62b8f37304b8a9a4bb54205347577, user.id=8012afab6cac4358ab878732e58d0689) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:05:11.423451 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b909c3d7-899f-4020-bb00-eddaa27509ec None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:05:11.424885 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b909c3d7-899f-4020-bb00-eddaa27509ec None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:05:11.430967 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 641/1287] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Jul 25 12:05:11 2022] GET /identity/v3/role_assignments?user.id=8012afab6cac4358ab878732e58d0689&scope.project.id=b9f62b8f37304b8a9a4bb54205347577&role.id=a7d20885b6d94928a34b3cc5b3a00006 => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:05:13.468931 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849be93c-ac74-4bcb-a95a-0507d8bf199f None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:05:13.469461 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849be93c-ac74-4bcb-a95a-0507d8bf199f None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:05:13.469552 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849be93c-ac74-4bcb-a95a-0507d8bf199f None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:05:13.470847 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 647/1288] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 25 12:05:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:05:13.476585 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1897b1d7-dbe6-4299-9fed-07378cafbc96 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:05:13.476955 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1897b1d7-dbe6-4299-9fed-07378cafbc96 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:05:13.477145 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1897b1d7-dbe6-4299-9fed-07378cafbc96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:05:13.495327 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-1897b1d7-dbe6-4299-9fed-07378cafbc96 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:05:13.496530 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-1897b1d7-dbe6-4299-9fed-07378cafbc96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:05:13.524438 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:05:13.562798 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 642/1289] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 25 12:05:13 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 12:05:13.590937 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849be93c-ac74-4bcb-a95a-0507d8bf199f None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:05:13.694845 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849be93c-ac74-4bcb-a95a-0507d8bf199f None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:05:13.705302 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:05:13.728863 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12925e39-89b3-4bd7-b9ae-66df168ea818 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:05:13.729593 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12925e39-89b3-4bd7-b9ae-66df168ea818 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:05:13.729877 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12925e39-89b3-4bd7-b9ae-66df168ea818 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:05:13.730153 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12925e39-89b3-4bd7-b9ae-66df168ea818 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:05:13.730803 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12925e39-89b3-4bd7-b9ae-66df168ea818 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:05:13.732198 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12925e39-89b3-4bd7-b9ae-66df168ea818 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:05:13.744941 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-12925e39-89b3-4bd7-b9ae-66df168ea818 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 25 12:05:13.744941 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:05:13.744941 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 12:05:13.744941 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 12:05:13.744941 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 12:05:13.744941 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 12:05:13.744941 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 12:05:13.744941 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 12:05:13.744941 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 25 12:05:13.744941 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jul 25 12:05:13.744941 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 12:05:13.744941 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jul 25 12:05:13.744941 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:05:13.744941 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:05:13.744941 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:05:13.744941 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:05:13.744941 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:05:13.746180 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:05:13.746180 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:05:13.746180 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:05:13.746180 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:05:13.746180 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:05:13.746180 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:05:13.746180 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 12:05:13.746180 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 12:05:13.746180 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 12:05:13.746180 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 12:05:13.746180 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:05:13.746180 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:05:13.746180 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:05:13.746180 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:05:13.746180 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:05:13.746180 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:05:13.747256 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:05:13.747256 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jul 25 12:05:13.747256 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:05:13.747256 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:05:13.747256 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:05:13.747256 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:05:13.747256 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:05:13.747256 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:05:13.747256 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:05:13.747256 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:05:13.747256 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 12:05:13.747256 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 12:05:13.747256 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 25 12:05:13.747256 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jul 25 12:05:13.747256 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 648/1290] 10.222.0.42 () {56 vars in 1117 bytes} [Mon Jul 25 12:05:13 2022] GET /identity/v3/services/placement => generated 90 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:05:13.752141 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1897b1d7-dbe6-4299-9fed-07378cafbc96 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:05:13.847935 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1897b1d7-dbe6-4299-9fed-07378cafbc96 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:05:13.880635 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebab2cd-15bb-40a2-a03c-27a70e15d4e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:05:13.881292 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebab2cd-15bb-40a2-a03c-27a70e15d4e1 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:05:13.881583 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebab2cd-15bb-40a2-a03c-27a70e15d4e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:05:13.881867 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebab2cd-15bb-40a2-a03c-27a70e15d4e1 None admin] PATH_INFO: `/v3/services` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:05:13.882579 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebab2cd-15bb-40a2-a03c-27a70e15d4e1 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:05:13.882893 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebab2cd-15bb-40a2-a03c-27a70e15d4e1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:05:13.884288 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebab2cd-15bb-40a2-a03c-27a70e15d4e1 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:05:13.889011 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 643/1291] 10.222.0.42 () {56 vars in 1116 bytes} [Mon Jul 25 12:05:13 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:05:13.894644 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12925e39-89b3-4bd7-b9ae-66df168ea818 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:05:13.998045 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12925e39-89b3-4bd7-b9ae-66df168ea818 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:05:14.030416 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bd9b46-d951-4cc9-aea8-fc9f5ba44bd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:05:14.031076 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bd9b46-d951-4cc9-aea8-fc9f5ba44bd5 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:05:14.031366 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bd9b46-d951-4cc9-aea8-fc9f5ba44bd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:05:14.031680 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bd9b46-d951-4cc9-aea8-fc9f5ba44bd5 None admin] PATH_INFO: `/v3/services` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:05:14.032379 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6bd9b46-d951-4cc9-aea8-fc9f5ba44bd5 None admin] RBAC: Adding query filter params (type=placement) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:05:14.032691 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6bd9b46-d951-4cc9-aea8-fc9f5ba44bd5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:05:14.034054 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6bd9b46-d951-4cc9-aea8-fc9f5ba44bd5 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:05:14.039287 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 649/1292] 10.222.0.42 () {56 vars in 1116 bytes} [Mon Jul 25 12:05:13 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:05:16.033278 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be734de-6c29-4b6c-b118-788529fce1f8 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:05:16.033781 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be734de-6c29-4b6c-b118-788529fce1f8 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:05:16.033781 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be734de-6c29-4b6c-b118-788529fce1f8 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:05:16.035167 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 644/1293] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 25 12:05:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:05:16.040529 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21602f10-37e2-4d49-9b0f-dd102ad1d349 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:05:16.040953 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21602f10-37e2-4d49-9b0f-dd102ad1d349 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:05:16.041101 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21602f10-37e2-4d49-9b0f-dd102ad1d349 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:05:16.058882 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-21602f10-37e2-4d49-9b0f-dd102ad1d349 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:05:16.060110 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-21602f10-37e2-4d49-9b0f-dd102ad1d349 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:05:16.090026 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:05:16.126667 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 650/1294] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 25 12:05:16 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 12:05:16.148990 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be734de-6c29-4b6c-b118-788529fce1f8 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:05:16.179433 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:05:16.247480 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be734de-6c29-4b6c-b118-788529fce1f8 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:05:16.297655 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2b58ad-b683-472f-bb94-69a60cedd530 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:05:16.298322 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2b58ad-b683-472f-bb94-69a60cedd530 None admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:05:16.298606 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2b58ad-b683-472f-bb94-69a60cedd530 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:05:16.298883 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2b58ad-b683-472f-bb94-69a60cedd530 None admin] PATH_INFO: `/v3/services` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:05:16.299639 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2b58ad-b683-472f-bb94-69a60cedd530 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:05:16.300999 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2b58ad-b683-472f-bb94-69a60cedd530 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:05:16.314615 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 645/1295] 10.222.0.42 () {60 vars in 1141 bytes} [Mon Jul 25 12:05:16 2022] POST /identity/v3/services => generated 246 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:05:18.370331 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a80991-71bb-433b-8318-703ac6e1eacb None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:05:18.370921 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a80991-71bb-433b-8318-703ac6e1eacb None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:05:18.370921 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a80991-71bb-433b-8318-703ac6e1eacb None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:05:18.372303 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 651/1296] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 25 12:05:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:05:18.377367 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fb3262-7f42-41bd-bc32-7aa47e7acf67 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:05:18.377794 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fb3262-7f42-41bd-bc32-7aa47e7acf67 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:05:18.377941 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fb3262-7f42-41bd-bc32-7aa47e7acf67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:05:18.396222 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-e4fb3262-7f42-41bd-bc32-7aa47e7acf67 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:05:18.397441 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-e4fb3262-7f42-41bd-bc32-7aa47e7acf67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:05:18.468251 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 646/1297] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 25 12:05:18 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 12:05:18.490584 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a80991-71bb-433b-8318-703ac6e1eacb None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:05:18.604667 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a80991-71bb-433b-8318-703ac6e1eacb None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:05:18.641111 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ac2ce3-c659-4723-b500-47476f8e79aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:05:18.641839 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ac2ce3-c659-4723-b500-47476f8e79aa None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:05:18.642129 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ac2ce3-c659-4723-b500-47476f8e79aa None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:05:18.642411 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ac2ce3-c659-4723-b500-47476f8e79aa None admin] PATH_INFO: `/v3/services/placement` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:05:18.643062 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ac2ce3-c659-4723-b500-47476f8e79aa None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:05:18.644478 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ac2ce3-c659-4723-b500-47476f8e79aa None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:05:18.656781 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-98ac2ce3-c659-4723-b500-47476f8e79aa None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 25 12:05:18.656781 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:05:18.656781 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 12:05:18.656781 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 12:05:18.656781 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 12:05:18.656781 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 12:05:18.656781 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 12:05:18.656781 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 12:05:18.656781 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 25 12:05:18.656781 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jul 25 12:05:18.656781 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 12:05:18.656781 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jul 25 12:05:18.656781 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:05:18.656781 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:05:18.656781 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:05:18.656781 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:05:18.656781 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:05:18.658011 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:05:18.658011 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:05:18.658011 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:05:18.658011 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:05:18.658011 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:05:18.658011 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:05:18.658011 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 12:05:18.658011 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 12:05:18.658011 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 12:05:18.658011 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 12:05:18.658011 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:05:18.658011 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:05:18.658011 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:05:18.658011 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:05:18.658011 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:05:18.658011 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:05:18.659111 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:05:18.659111 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jul 25 12:05:18.659111 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:05:18.659111 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:05:18.659111 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:05:18.659111 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:05:18.659111 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:05:18.659111 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:05:18.659111 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:05:18.659111 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:05:18.659111 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 12:05:18.659111 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 12:05:18.659111 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 25 12:05:18.659111 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jul 25 12:05:18.659111 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 652/1298] 10.222.0.42 () {56 vars in 1117 bytes} [Mon Jul 25 12:05:18 2022] GET /identity/v3/services/placement => generated 90 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:05:18.665028 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fb3262-7f42-41bd-bc32-7aa47e7acf67 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:05:18.684611 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:05:18.768610 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fb3262-7f42-41bd-bc32-7aa47e7acf67 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:05:18.804045 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86646e3b-276f-45cb-978d-31b62b72f55c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:05:18.804715 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86646e3b-276f-45cb-978d-31b62b72f55c None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:05:18.805000 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86646e3b-276f-45cb-978d-31b62b72f55c None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:05:18.805283 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86646e3b-276f-45cb-978d-31b62b72f55c None admin] PATH_INFO: `/v3/services` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:05:18.806002 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86646e3b-276f-45cb-978d-31b62b72f55c None admin] RBAC: Adding query filter params (name=placement) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:05:18.806313 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86646e3b-276f-45cb-978d-31b62b72f55c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:05:18.807714 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86646e3b-276f-45cb-978d-31b62b72f55c None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:05:18.812920 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 647/1299] 10.222.0.42 () {56 vars in 1116 bytes} [Mon Jul 25 12:05:18 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:05:18.819382 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ac2ce3-c659-4723-b500-47476f8e79aa None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:05:18.851159 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:05:18.926009 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ac2ce3-c659-4723-b500-47476f8e79aa None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:05:18.959161 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b101a70-42be-49e2-96cc-be1b3d470856 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:05:18.960396 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b101a70-42be-49e2-96cc-be1b3d470856 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:05:18.960689 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b101a70-42be-49e2-96cc-be1b3d470856 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:05:18.960973 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b101a70-42be-49e2-96cc-be1b3d470856 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:05:18.961765 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b101a70-42be-49e2-96cc-be1b3d470856 None admin] RBAC: Adding query filter params (interface=public, service_id=f2a6766a8a904a36a0043f1f52508d02, region_id=RegionOne) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:05:18.962087 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b101a70-42be-49e2-96cc-be1b3d470856 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:05:18.963446 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b101a70-42be-49e2-96cc-be1b3d470856 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:05:18.968421 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 653/1300] 10.222.0.42 () {56 vars in 1251 bytes} [Mon Jul 25 12:05:18 2022] GET /identity/v3/endpoints?service_id=f2a6766a8a904a36a0043f1f52508d02&interface=public®ion_id=RegionOne => generated 194 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:05:18.974095 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86646e3b-276f-45cb-978d-31b62b72f55c None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:05:19.053705 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:05:19.071799 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86646e3b-276f-45cb-978d-31b62b72f55c None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:05:19.104595 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f576d73c-5258-4a16-a955-3ce3a0185be3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:05:19.105252 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f576d73c-5258-4a16-a955-3ce3a0185be3 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:05:19.105538 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f576d73c-5258-4a16-a955-3ce3a0185be3 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:05:19.105816 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f576d73c-5258-4a16-a955-3ce3a0185be3 None admin] PATH_INFO: `/v3/services` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:05:19.106494 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f576d73c-5258-4a16-a955-3ce3a0185be3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:05:19.107879 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f576d73c-5258-4a16-a955-3ce3a0185be3 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:05:19.114494 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 648/1301] 10.222.0.42 () {56 vars in 1087 bytes} [Mon Jul 25 12:05:18 2022] GET /identity/v3/services => generated 2191 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:05:21.148921 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b73114b-3d6b-4015-a871-a0569a904ee7 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:05:21.149473 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b73114b-3d6b-4015-a871-a0569a904ee7 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:05:21.149473 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b73114b-3d6b-4015-a871-a0569a904ee7 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:05:21.150807 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 654/1302] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 25 12:05:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:05:21.155816 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4d1e65-0e72-45bf-8d14-53de63703f38 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:05:21.156234 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4d1e65-0e72-45bf-8d14-53de63703f38 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:05:21.156386 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4d1e65-0e72-45bf-8d14-53de63703f38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:05:21.174706 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-ee4d1e65-0e72-45bf-8d14-53de63703f38 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:05:21.175919 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-ee4d1e65-0e72-45bf-8d14-53de63703f38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:05:21.239172 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 649/1303] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 25 12:05:21 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 12:05:21.262373 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b73114b-3d6b-4015-a871-a0569a904ee7 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:05:21.355092 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:05:21.361938 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b73114b-3d6b-4015-a871-a0569a904ee7 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:05:21.395261 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255de726-6990-4b72-9bc3-bdb726052d8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:05:21.396067 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255de726-6990-4b72-9bc3-bdb726052d8b None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:05:21.396357 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255de726-6990-4b72-9bc3-bdb726052d8b None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:05:21.396658 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255de726-6990-4b72-9bc3-bdb726052d8b None admin] PATH_INFO: `/v3/services/placement` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:05:21.397307 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255de726-6990-4b72-9bc3-bdb726052d8b None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:05:21.398655 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255de726-6990-4b72-9bc3-bdb726052d8b None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:05:21.410535 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-255de726-6990-4b72-9bc3-bdb726052d8b None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 25 12:05:21.410535 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:05:21.410535 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 12:05:21.410535 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 12:05:21.410535 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 12:05:21.410535 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 12:05:21.410535 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 12:05:21.410535 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 12:05:21.410535 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 25 12:05:21.410535 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jul 25 12:05:21.410535 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 12:05:21.410535 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jul 25 12:05:21.410535 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 12:05:21.410535 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 12:05:21.410535 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 12:05:21.410535 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 12:05:21.410535 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 12:05:21.411719 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 12:05:21.411719 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 12:05:21.411719 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 12:05:21.411719 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 12:05:21.411719 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 12:05:21.411719 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 12:05:21.411719 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 12:05:21.411719 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 12:05:21.411719 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 12:05:21.411719 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 12:05:21.411719 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 12:05:21.411719 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 12:05:21.411719 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 12:05:21.411719 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 12:05:21.411719 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 12:05:21.411719 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 12:05:21.412766 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 12:05:21.412766 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jul 25 12:05:21.412766 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 12:05:21.412766 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jul 25 12:05:21.412766 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 12:05:21.412766 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 12:05:21.412766 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 12:05:21.412766 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jul 25 12:05:21.412766 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 12:05:21.412766 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jul 25 12:05:21.412766 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 12:05:21.412766 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 12:05:21.412766 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 25 12:05:21.412766 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jul 25 12:05:21.412766 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 655/1304] 10.222.0.42 () {56 vars in 1117 bytes} [Mon Jul 25 12:05:21 2022] GET /identity/v3/services/placement => generated 90 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 12:05:21.417697 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4d1e65-0e72-45bf-8d14-53de63703f38 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:05:21.491197 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:05:21.513652 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4d1e65-0e72-45bf-8d14-53de63703f38 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:05:21.547488 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991acf8e-0443-4bb1-b653-40f35d8d0221 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:05:21.548173 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991acf8e-0443-4bb1-b653-40f35d8d0221 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:05:21.548483 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991acf8e-0443-4bb1-b653-40f35d8d0221 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:05:21.548863 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991acf8e-0443-4bb1-b653-40f35d8d0221 None admin] PATH_INFO: `/v3/services` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:05:21.549571 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991acf8e-0443-4bb1-b653-40f35d8d0221 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:05:21.549919 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991acf8e-0443-4bb1-b653-40f35d8d0221 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:05:21.551459 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991acf8e-0443-4bb1-b653-40f35d8d0221 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:05:21.557176 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 650/1305] 10.222.0.42 () {56 vars in 1116 bytes} [Mon Jul 25 12:05:21 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:05:21.562872 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255de726-6990-4b72-9bc3-bdb726052d8b None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:05:21.661386 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255de726-6990-4b72-9bc3-bdb726052d8b None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:05:21.694821 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcafb9d4-748c-4fc1-aede-5991814dff5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:05:21.695463 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcafb9d4-748c-4fc1-aede-5991814dff5c None admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:05:21.695782 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcafb9d4-748c-4fc1-aede-5991814dff5c None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:05:21.696065 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcafb9d4-748c-4fc1-aede-5991814dff5c None admin] PATH_INFO: `/v3/endpoints` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:05:21.696809 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcafb9d4-748c-4fc1-aede-5991814dff5c None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:05:21.698166 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcafb9d4-748c-4fc1-aede-5991814dff5c None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:05:21.716891 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 656/1306] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Jul 25 12:05:21 2022] POST /identity/v3/endpoints => generated 331 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:05:31.554389 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=30 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:05:31.712659 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:06:24.170398 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a9cf28-20c5-4677-8540-477acb3126e8 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:06:24.170968 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a9cf28-20c5-4677-8540-477acb3126e8 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:06:24.170968 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a9cf28-20c5-4677-8540-477acb3126e8 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:06:24.175965 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 651/1307] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 25 12:06:24 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:06:24.177503 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392116e6-f457-4836-9f88-ac38dac53835 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:06:24.177933 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392116e6-f457-4836-9f88-ac38dac53835 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:06:24.178119 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392116e6-f457-4836-9f88-ac38dac53835 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:06:24.264959 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-392116e6-f457-4836-9f88-ac38dac53835 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:06:24.266124 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-392116e6-f457-4836-9f88-ac38dac53835 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:06:24.452656 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 657/1308] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 25 12:06:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 276 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 12:06:24.505386 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb04001-5e96-4a26-a36d-f64b448f6a94 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:06:24.505753 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb04001-5e96-4a26-a36d-f64b448f6a94 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:06:24.506175 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb04001-5e96-4a26-a36d-f64b448f6a94 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:06:24.508106 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 652/1309] 10.222.0.42 () {54 vars in 965 bytes} [Mon Jul 25 12:06:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 12:06:24.517417 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e26d92b-c225-4e76-b075-d855da39b037 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:06:24.517802 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e26d92b-c225-4e76-b075-d855da39b037 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:06:24.518290 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e26d92b-c225-4e76-b075-d855da39b037 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:06:24.624778 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-7e26d92b-c225-4e76-b075-d855da39b037 None None] MFA Rules not processed for user `51733781201243e8abf9eca39b207750`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:06:24.626872 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-7e26d92b-c225-4e76-b075-d855da39b037 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:06:24.656546 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:06:24.748343 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 658/1310] 10.222.0.42 () {58 vars in 1062 bytes} [Mon Jul 25 12:06:24 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:06:24.757543 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb04001-5e96-4a26-a36d-f64b448f6a94 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:06:24.759682 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-dcb04001-5e96-4a26-a36d-f64b448f6a94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:06:24.896457 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb04001-5e96-4a26-a36d-f64b448f6a94 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:06:24.971931 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b581c9-482c-4963-8ad4-3884197cbb15 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51733781201243e8abf9eca39b207750', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:06:24.972483 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b581c9-482c-4963-8ad4-3884197cbb15 service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:06:24.972771 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b581c9-482c-4963-8ad4-3884197cbb15 service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:06:24.973068 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b581c9-482c-4963-8ad4-3884197cbb15 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:06:24.974702 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-34b581c9-482c-4963-8ad4-3884197cbb15 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:06:25.037580 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:06:25.102426 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b581c9-482c-4963-8ad4-3884197cbb15 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:06:25.104311 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b581c9-482c-4963-8ad4-3884197cbb15 service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:06:25.126312 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 653/1311] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:06:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:06:27.461270 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1c7920-c074-4221-9c37-7b490e3102c9 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:06:27.461792 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1c7920-c074-4221-9c37-7b490e3102c9 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:06:27.461871 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1c7920-c074-4221-9c37-7b490e3102c9 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:06:27.463381 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 659/1312] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 25 12:06:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:06:27.468389 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e93709-5d10-428f-b041-3763a9d0b3ec None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:06:27.468801 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e93709-5d10-428f-b041-3763a9d0b3ec None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:06:27.469110 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e93709-5d10-428f-b041-3763a9d0b3ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:06:27.540569 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-23e93709-5d10-428f-b041-3763a9d0b3ec None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:06:27.541694 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-23e93709-5d10-428f-b041-3763a9d0b3ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:06:27.637993 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 654/1313] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 25 12:06:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:06:27.677179 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1c7920-c074-4221-9c37-7b490e3102c9 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:06:27.678785 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-bc1c7920-c074-4221-9c37-7b490e3102c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:06:27.796281 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1c7920-c074-4221-9c37-7b490e3102c9 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:06:27.874836 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:06:27.877638 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2065b5-cde9-4f76-8ced-7d36eff731e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51733781201243e8abf9eca39b207750', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:06:27.878326 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2065b5-cde9-4f76-8ced-7d36eff731e3 service neutron] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:06:27.878954 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2065b5-cde9-4f76-8ced-7d36eff731e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:06:27.879695 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2065b5-cde9-4f76-8ced-7d36eff731e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:06:27.881609 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-fa2065b5-cde9-4f76-8ced-7d36eff731e3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:06:27.994726 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2065b5-cde9-4f76-8ced-7d36eff731e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:06:27.996192 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2065b5-cde9-4f76-8ced-7d36eff731e3 service neutron] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:06:28.014281 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 660/1314] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:06:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:06:30.302586 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671f9813-94e1-431d-bf97-6a828542e613 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:06:30.303136 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671f9813-94e1-431d-bf97-6a828542e613 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:06:30.303136 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671f9813-94e1-431d-bf97-6a828542e613 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:06:30.304550 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 655/1315] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 25 12:06:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:06:30.309663 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64cb990-27bd-4462-8649-234803e4007d None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:06:30.310106 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64cb990-27bd-4462-8649-234803e4007d None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:06:30.310231 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64cb990-27bd-4462-8649-234803e4007d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:06:30.376616 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-a64cb990-27bd-4462-8649-234803e4007d None None] MFA Rules not processed for user `b66651fed8eb43758d5d02472aa1ffaf`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:06:30.377771 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-a64cb990-27bd-4462-8649-234803e4007d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:06:30.428736 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:06:30.473481 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 661/1316] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 25 12:06:30 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:06:30.512948 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671f9813-94e1-431d-bf97-6a828542e613 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:06:30.514472 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-671f9813-94e1-431d-bf97-6a828542e613 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:06:30.524267 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:06:30.616403 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671f9813-94e1-431d-bf97-6a828542e613 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:06:30.687501 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0b6188-df42-4be3-8766-a59675a25400 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51733781201243e8abf9eca39b207750', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:06:30.688641 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0b6188-df42-4be3-8766-a59675a25400 service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:06:30.688924 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0b6188-df42-4be3-8766-a59675a25400 service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:06:30.689204 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0b6188-df42-4be3-8766-a59675a25400 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:06:30.690848 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-ec0b6188-df42-4be3-8766-a59675a25400 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:06:30.753370 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:06:30.800886 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0b6188-df42-4be3-8766-a59675a25400 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:06:30.802333 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0b6188-df42-4be3-8766-a59675a25400 service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:06:30.820071 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 656/1317] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:06:30 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:06:33.403450 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1455598-8427-4faf-9ab5-42ec70081184 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:06:33.404045 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1455598-8427-4faf-9ab5-42ec70081184 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:06:33.404045 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1455598-8427-4faf-9ab5-42ec70081184 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:06:33.405429 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 662/1318] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 25 12:06:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:06:33.411513 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89929278-eaae-45ea-bb83-086e842a15f9 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:06:33.411914 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89929278-eaae-45ea-bb83-086e842a15f9 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:06:33.412105 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89929278-eaae-45ea-bb83-086e842a15f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:06:33.477379 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-89929278-eaae-45ea-bb83-086e842a15f9 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:06:33.478593 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-89929278-eaae-45ea-bb83-086e842a15f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:06:33.574778 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 657/1319] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 25 12:06:33 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:06:33.625725 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b08ba01-ccd1-4520-8f40-de62fc29eb44 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:06:33.626022 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b08ba01-ccd1-4520-8f40-de62fc29eb44 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:06:33.626313 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b08ba01-ccd1-4520-8f40-de62fc29eb44 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:06:33.627754 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 663/1320] 10.222.0.42 () {54 vars in 965 bytes} [Mon Jul 25 12:06:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:06:33.633371 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510df316-617f-46c8-8133-21276ed79af8 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:06:33.633668 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510df316-617f-46c8-8133-21276ed79af8 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:06:33.633954 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510df316-617f-46c8-8133-21276ed79af8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:06:33.657102 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:06:33.711053 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-510df316-617f-46c8-8133-21276ed79af8 None None] MFA Rules not processed for user `51733781201243e8abf9eca39b207750`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:06:33.712304 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-510df316-617f-46c8-8133-21276ed79af8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:06:33.787587 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 658/1321] 10.222.0.42 () {58 vars in 1062 bytes} [Mon Jul 25 12:06:33 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:06:33.796521 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b08ba01-ccd1-4520-8f40-de62fc29eb44 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:06:33.798071 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-1b08ba01-ccd1-4520-8f40-de62fc29eb44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:06:33.899275 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b08ba01-ccd1-4520-8f40-de62fc29eb44 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:06:33.972178 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78d4d24-7178-4e99-80cc-d1d2f41456dd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51733781201243e8abf9eca39b207750', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:06:33.972756 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78d4d24-7178-4e99-80cc-d1d2f41456dd service neutron] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:06:33.973047 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78d4d24-7178-4e99-80cc-d1d2f41456dd service neutron] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:06:33.973332 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78d4d24-7178-4e99-80cc-d1d2f41456dd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:06:33.974949 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-a78d4d24-7178-4e99-80cc-d1d2f41456dd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:06:33.982181 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:06:34.082680 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78d4d24-7178-4e99-80cc-d1d2f41456dd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:06:34.084158 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78d4d24-7178-4e99-80cc-d1d2f41456dd service neutron] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:06:34.103312 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 664/1322] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:06:33 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:06:37.196608 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9f1294-ba70-464c-9978-8b368bd972a9 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:06:37.197141 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9f1294-ba70-464c-9978-8b368bd972a9 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:06:37.197141 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9f1294-ba70-464c-9978-8b368bd972a9 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:06:37.198532 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 659/1323] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 25 12:06:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:06:37.204262 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2e03d4-3a53-4c4e-86c5-dbdeffcb5873 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:06:37.204787 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2e03d4-3a53-4c4e-86c5-dbdeffcb5873 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:06:37.205197 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2e03d4-3a53-4c4e-86c5-dbdeffcb5873 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:06:37.270823 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-aa2e03d4-3a53-4c4e-86c5-dbdeffcb5873 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:06:37.272093 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-aa2e03d4-3a53-4c4e-86c5-dbdeffcb5873 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:06:37.332418 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:06:37.373016 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 665/1324] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 25 12:06:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 12:06:37.409979 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9f1294-ba70-464c-9978-8b368bd972a9 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:06:37.411629 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-2a9f1294-ba70-464c-9978-8b368bd972a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:06:37.523345 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9f1294-ba70-464c-9978-8b368bd972a9 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:06:37.536811 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:06:37.604614 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933be999-1ac1-4e0a-8f88-9b1d6711a92b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51733781201243e8abf9eca39b207750', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:06:37.605264 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933be999-1ac1-4e0a-8f88-9b1d6711a92b service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:06:37.605554 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933be999-1ac1-4e0a-8f88-9b1d6711a92b service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:06:37.605831 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933be999-1ac1-4e0a-8f88-9b1d6711a92b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:06:37.607484 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-933be999-1ac1-4e0a-8f88-9b1d6711a92b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:06:37.715513 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933be999-1ac1-4e0a-8f88-9b1d6711a92b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:06:37.716984 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933be999-1ac1-4e0a-8f88-9b1d6711a92b service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:06:37.734770 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 660/1325] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:06:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:06:39.810346 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540a39dd-50be-43ca-8227-b998e3fa7d59 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:06:39.810883 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540a39dd-50be-43ca-8227-b998e3fa7d59 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:06:39.810989 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540a39dd-50be-43ca-8227-b998e3fa7d59 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:06:39.812332 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 666/1326] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 25 12:06:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:06:39.817646 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e94e0d-1e77-4374-938e-11dfdeb1db85 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:06:39.818054 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e94e0d-1e77-4374-938e-11dfdeb1db85 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:06:39.818240 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e94e0d-1e77-4374-938e-11dfdeb1db85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:06:39.866825 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:06:39.884766 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-87e94e0d-1e77-4374-938e-11dfdeb1db85 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:06:39.885980 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-87e94e0d-1e77-4374-938e-11dfdeb1db85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:06:39.981223 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 661/1327] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 25 12:06:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:06:40.017182 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540a39dd-50be-43ca-8227-b998e3fa7d59 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:06:40.018724 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-540a39dd-50be-43ca-8227-b998e3fa7d59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:06:40.122720 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540a39dd-50be-43ca-8227-b998e3fa7d59 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:06:40.196207 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c80a60a-c11f-4fc7-b878-0a688b935769 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51733781201243e8abf9eca39b207750', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:06:40.196863 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c80a60a-c11f-4fc7-b878-0a688b935769 service neutron] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:06:40.197153 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c80a60a-c11f-4fc7-b878-0a688b935769 service neutron] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:06:40.197434 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c80a60a-c11f-4fc7-b878-0a688b935769 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:06:40.199071 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-8c80a60a-c11f-4fc7-b878-0a688b935769 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:06:40.216152 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:06:40.313129 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c80a60a-c11f-4fc7-b878-0a688b935769 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:06:40.314568 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c80a60a-c11f-4fc7-b878-0a688b935769 service neutron] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:06:40.333481 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 667/1328] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:06:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:06:42.407986 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92797242-18cc-40c5-97c7-8daffbf12e6f None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:06:42.408509 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92797242-18cc-40c5-97c7-8daffbf12e6f None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:06:42.408509 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92797242-18cc-40c5-97c7-8daffbf12e6f None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:06:42.409876 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 662/1329] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 25 12:06:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:06:42.415183 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b7d2d6-3944-4286-b1ab-9de7f7fa6bf1 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:06:42.415623 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b7d2d6-3944-4286-b1ab-9de7f7fa6bf1 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:06:42.415792 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b7d2d6-3944-4286-b1ab-9de7f7fa6bf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:06:42.483963 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-c5b7d2d6-3944-4286-b1ab-9de7f7fa6bf1 None None] MFA Rules not processed for user `b66651fed8eb43758d5d02472aa1ffaf`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:06:42.485177 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-c5b7d2d6-3944-4286-b1ab-9de7f7fa6bf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:06:42.550259 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:06:42.583977 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 668/1330] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 25 12:06:42 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:06:42.621163 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92797242-18cc-40c5-97c7-8daffbf12e6f None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:06:42.622708 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-92797242-18cc-40c5-97c7-8daffbf12e6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:06:42.737508 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92797242-18cc-40c5-97c7-8daffbf12e6f None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:06:42.758902 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:06:42.821086 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b99ab2-3371-4c62-a124-345e47275d63 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51733781201243e8abf9eca39b207750', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:06:42.821774 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b99ab2-3371-4c62-a124-345e47275d63 service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:06:42.822065 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b99ab2-3371-4c62-a124-345e47275d63 service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:06:42.822348 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b99ab2-3371-4c62-a124-345e47275d63 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:06:42.824043 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-97b99ab2-3371-4c62-a124-345e47275d63 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:06:42.935478 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b99ab2-3371-4c62-a124-345e47275d63 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:06:42.938140 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b99ab2-3371-4c62-a124-345e47275d63 service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:06:42.956527 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 663/1331] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:06:42 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:06:45.248682 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54cc8c3-66e3-4fa4-84b6-f8cd83b71bbe None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:06:45.249735 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54cc8c3-66e3-4fa4-84b6-f8cd83b71bbe None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:06:45.249832 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54cc8c3-66e3-4fa4-84b6-f8cd83b71bbe None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:06:45.251120 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 669/1332] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 25 12:06:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:06:45.256227 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec9cbb9-93d9-48d4-8b81-0bfd58a418ba None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:06:45.256654 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec9cbb9-93d9-48d4-8b81-0bfd58a418ba None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:06:45.256819 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec9cbb9-93d9-48d4-8b81-0bfd58a418ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:06:45.323938 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:06:45.328145 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-8ec9cbb9-93d9-48d4-8b81-0bfd58a418ba None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:06:45.329769 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-8ec9cbb9-93d9-48d4-8b81-0bfd58a418ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:06:45.425489 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 664/1333] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 25 12:06:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:06:45.461852 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54cc8c3-66e3-4fa4-84b6-f8cd83b71bbe None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:06:45.463395 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-b54cc8c3-66e3-4fa4-84b6-f8cd83b71bbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:06:45.566597 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54cc8c3-66e3-4fa4-84b6-f8cd83b71bbe None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:06:45.638021 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484c7269-0445-4359-a8f9-a586bc560419 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51733781201243e8abf9eca39b207750', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:06:45.638666 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484c7269-0445-4359-a8f9-a586bc560419 service neutron] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:06:45.638975 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484c7269-0445-4359-a8f9-a586bc560419 service neutron] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:06:45.639242 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484c7269-0445-4359-a8f9-a586bc560419 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:06:45.640940 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-484c7269-0445-4359-a8f9-a586bc560419 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:06:45.667245 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:06:45.749026 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484c7269-0445-4359-a8f9-a586bc560419 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:06:45.750457 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484c7269-0445-4359-a8f9-a586bc560419 service neutron] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:06:45.768192 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 670/1334] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:06:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:06:47.835338 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a45fb1-2a0f-446c-87f2-5802151aee81 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:06:47.835869 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a45fb1-2a0f-446c-87f2-5802151aee81 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:06:47.835961 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a45fb1-2a0f-446c-87f2-5802151aee81 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:06:47.837313 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 665/1335] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 25 12:06:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:06:47.843084 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e70be6-84e9-485b-90f5-e538ed1b2690 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:06:47.843583 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e70be6-84e9-485b-90f5-e538ed1b2690 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:06:47.843717 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e70be6-84e9-485b-90f5-e538ed1b2690 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:06:47.913628 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-f7e70be6-84e9-485b-90f5-e538ed1b2690 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:06:47.914877 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-f7e70be6-84e9-485b-90f5-e538ed1b2690 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:06:47.988047 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:06:48.011816 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 671/1336] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 25 12:06:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 12:06:48.051922 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a45fb1-2a0f-446c-87f2-5802151aee81 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:06:48.053468 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-38a45fb1-2a0f-446c-87f2-5802151aee81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:06:48.162949 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a45fb1-2a0f-446c-87f2-5802151aee81 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:06:48.190869 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:06:48.235193 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05e522d-99bb-45ef-ba15-c00ef8f3dac0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51733781201243e8abf9eca39b207750', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:06:48.235897 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05e522d-99bb-45ef-ba15-c00ef8f3dac0 service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:06:48.236189 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05e522d-99bb-45ef-ba15-c00ef8f3dac0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:06:48.236475 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05e522d-99bb-45ef-ba15-c00ef8f3dac0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:06:48.238127 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-f05e522d-99bb-45ef-ba15-c00ef8f3dac0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:06:48.345164 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05e522d-99bb-45ef-ba15-c00ef8f3dac0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:06:48.346533 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05e522d-99bb-45ef-ba15-c00ef8f3dac0 service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:06:48.364741 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 666/1337] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:06:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:06:50.722832 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc1b5ef-b90d-410c-89b1-0ee4fd60a343 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:06:50.723423 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc1b5ef-b90d-410c-89b1-0ee4fd60a343 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:06:50.723423 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc1b5ef-b90d-410c-89b1-0ee4fd60a343 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:06:50.724815 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 672/1338] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 25 12:06:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:06:50.730031 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0a3a1c-c3c6-4da4-bbe2-f03b91b89a57 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:06:50.730481 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0a3a1c-c3c6-4da4-bbe2-f03b91b89a57 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:06:50.730584 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0a3a1c-c3c6-4da4-bbe2-f03b91b89a57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:06:50.798425 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-fb0a3a1c-c3c6-4da4-bbe2-f03b91b89a57 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:06:50.799674 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-fb0a3a1c-c3c6-4da4-bbe2-f03b91b89a57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:06:50.807823 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:06:50.896617 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 667/1339] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 25 12:06:50 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:06:50.932594 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc1b5ef-b90d-410c-89b1-0ee4fd60a343 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:06:50.934124 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-3dc1b5ef-b90d-410c-89b1-0ee4fd60a343 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:06:51.039463 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc1b5ef-b90d-410c-89b1-0ee4fd60a343 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:06:51.111274 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b089b7-a49f-4be7-ac25-139f4991427d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51733781201243e8abf9eca39b207750', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:06:51.111954 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b089b7-a49f-4be7-ac25-139f4991427d service neutron] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:06:51.112250 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b089b7-a49f-4be7-ac25-139f4991427d service neutron] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:06:51.112536 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b089b7-a49f-4be7-ac25-139f4991427d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:06:51.114157 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-23b089b7-a49f-4be7-ac25-139f4991427d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:06:51.149190 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:06:51.222598 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b089b7-a49f-4be7-ac25-139f4991427d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:06:51.224156 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b089b7-a49f-4be7-ac25-139f4991427d service neutron] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:06:51.242957 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 673/1340] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:06:50 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:06:55.242912 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf03d191-1cce-4629-a2ba-23c7c7b46604 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:06:55.243287 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf03d191-1cce-4629-a2ba-23c7c7b46604 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:06:55.243482 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf03d191-1cce-4629-a2ba-23c7c7b46604 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:06:55.245043 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 668/1341] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 25 12:06:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:06:55.250592 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e91b6b-8133-4c90-b202-92f4e373eb0b None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:06:55.251015 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e91b6b-8133-4c90-b202-92f4e373eb0b None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:06:55.251225 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e91b6b-8133-4c90-b202-92f4e373eb0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:06:55.317380 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-16e91b6b-8133-4c90-b202-92f4e373eb0b None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:06:55.318605 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-16e91b6b-8133-4c90-b202-92f4e373eb0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:06:55.406843 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:06:55.418691 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 674/1342] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 25 12:06:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 12:06:55.457729 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf03d191-1cce-4629-a2ba-23c7c7b46604 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:06:55.459212 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-cf03d191-1cce-4629-a2ba-23c7c7b46604 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:06:55.566183 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf03d191-1cce-4629-a2ba-23c7c7b46604 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:06:55.604871 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:06:55.641439 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23296332-dc27-4cdf-b376-b87ab6637fe6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51733781201243e8abf9eca39b207750', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:06:55.642103 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23296332-dc27-4cdf-b376-b87ab6637fe6 service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:06:55.642397 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23296332-dc27-4cdf-b376-b87ab6637fe6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:06:55.642679 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23296332-dc27-4cdf-b376-b87ab6637fe6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:06:55.644365 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-23296332-dc27-4cdf-b376-b87ab6637fe6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:06:55.756205 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23296332-dc27-4cdf-b376-b87ab6637fe6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:06:55.757685 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23296332-dc27-4cdf-b376-b87ab6637fe6 service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:06:55.777499 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 669/1343] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:06:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:06:58.389927 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2921c1-6434-41d9-a1cc-0324242293ec None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:06:58.390455 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2921c1-6434-41d9-a1cc-0324242293ec None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:06:58.390455 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2921c1-6434-41d9-a1cc-0324242293ec None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:06:58.391781 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 675/1344] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 25 12:06:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:06:58.396939 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0d062f-918c-4ed1-8d8c-773ee1880b16 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:06:58.397327 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0d062f-918c-4ed1-8d8c-773ee1880b16 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:06:58.397489 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0d062f-918c-4ed1-8d8c-773ee1880b16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:06:58.476189 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-9a0d062f-918c-4ed1-8d8c-773ee1880b16 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:06:58.477477 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-9a0d062f-918c-4ed1-8d8c-773ee1880b16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:06:58.498276 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:06:58.594830 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 670/1345] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 25 12:06:58 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:06:58.633028 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2921c1-6434-41d9-a1cc-0324242293ec None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:06:58.634801 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-0d2921c1-6434-41d9-a1cc-0324242293ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:06:58.741120 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2921c1-6434-41d9-a1cc-0324242293ec None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:06:58.820165 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781728d6-46d1-4acb-b65d-a20e6c4aeaaf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51733781201243e8abf9eca39b207750', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:06:58.820820 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781728d6-46d1-4acb-b65d-a20e6c4aeaaf service neutron] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:06:58.821106 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781728d6-46d1-4acb-b65d-a20e6c4aeaaf service neutron] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:06:58.821381 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781728d6-46d1-4acb-b65d-a20e6c4aeaaf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:06:58.823018 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-781728d6-46d1-4acb-b65d-a20e6c4aeaaf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:06:58.866220 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:06:58.933182 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781728d6-46d1-4acb-b65d-a20e6c4aeaaf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:06:58.934621 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781728d6-46d1-4acb-b65d-a20e6c4aeaaf service neutron] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:06:58.953604 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 676/1346] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:06:58 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:07:02.949905 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e43d46-955c-4195-a901-2c522b369184 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:07:02.950449 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e43d46-955c-4195-a901-2c522b369184 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:07:02.950547 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e43d46-955c-4195-a901-2c522b369184 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:07:02.951983 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 671/1347] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 25 12:07:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:07:02.957398 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed512ec5-ffe4-45e1-adbd-ed48251e692c None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:07:02.957844 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed512ec5-ffe4-45e1-adbd-ed48251e692c None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:07:02.957956 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed512ec5-ffe4-45e1-adbd-ed48251e692c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:07:03.023881 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-ed512ec5-ffe4-45e1-adbd-ed48251e692c None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:07:03.025109 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-ed512ec5-ffe4-45e1-adbd-ed48251e692c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:07:03.125525 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:07:03.143908 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 677/1348] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 25 12:07:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:07:03.209722 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e43d46-955c-4195-a901-2c522b369184 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:07:03.211334 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-17e43d46-955c-4195-a901-2c522b369184 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:07:03.328554 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e43d46-955c-4195-a901-2c522b369184 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:07:03.377834 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:07:03.406273 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e78f712-7b9e-4f5c-a9cc-2bf1b5403829 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51733781201243e8abf9eca39b207750', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:07:03.406928 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e78f712-7b9e-4f5c-a9cc-2bf1b5403829 service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:07:03.407219 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e78f712-7b9e-4f5c-a9cc-2bf1b5403829 service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:07:03.407498 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e78f712-7b9e-4f5c-a9cc-2bf1b5403829 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:07:03.409203 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-0e78f712-7b9e-4f5c-a9cc-2bf1b5403829 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:07:03.527089 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e78f712-7b9e-4f5c-a9cc-2bf1b5403829 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:07:03.529220 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e78f712-7b9e-4f5c-a9cc-2bf1b5403829 service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:07:03.549816 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 672/1349] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:07:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:07:08.504577 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81e1a83-5dc4-45cc-b4a4-5fc54ac5e305 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:07:08.505585 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81e1a83-5dc4-45cc-b4a4-5fc54ac5e305 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:07:08.506032 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81e1a83-5dc4-45cc-b4a4-5fc54ac5e305 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:07:08.507850 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 678/1350] 10.222.0.42 () {54 vars in 921 bytes} [Mon Jul 25 12:07:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:07:08.514075 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536a9135-b762-42b6-9f4c-f7c6cde68940 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:07:08.514855 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536a9135-b762-42b6-9f4c-f7c6cde68940 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:07:08.515289 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536a9135-b762-42b6-9f4c-f7c6cde68940 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:07:08.598699 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-536a9135-b762-42b6-9f4c-f7c6cde68940 None None] MFA Rules not processed for user `8012afab6cac4358ab878732e58d0689`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:07:08.600180 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-536a9135-b762-42b6-9f4c-f7c6cde68940 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:07:08.619813 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:07:08.734500 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 673/1351] 10.222.0.42 () {58 vars in 1018 bytes} [Mon Jul 25 12:07:08 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 12:07:12.693107 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b6bcd5-c996-4355-8f9f-eb1923d4e336 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:07:12.693628 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b6bcd5-c996-4355-8f9f-eb1923d4e336 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:07:12.693728 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b6bcd5-c996-4355-8f9f-eb1923d4e336 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:07:12.695026 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 679/1352] 10.222.0.42 () {54 vars in 921 bytes} [Mon Jul 25 12:07:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:07:12.700971 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8661ad5-f747-48b2-b8fc-f22459456d65 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:07:12.701440 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8661ad5-f747-48b2-b8fc-f22459456d65 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:07:12.701536 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8661ad5-f747-48b2-b8fc-f22459456d65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:07:12.788220 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-a8661ad5-f747-48b2-b8fc-f22459456d65 None None] MFA Rules not processed for user `8012afab6cac4358ab878732e58d0689`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:07:12.788220 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-a8661ad5-f747-48b2-b8fc-f22459456d65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:07:12.860526 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:07:12.898307 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 674/1353] 10.222.0.42 () {58 vars in 1018 bytes} [Mon Jul 25 12:07:12 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:07:14.306454 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c4ebdd-fa73-44ea-a145-3640064f806b None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:07:14.306962 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c4ebdd-fa73-44ea-a145-3640064f806b None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:07:14.307075 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c4ebdd-fa73-44ea-a145-3640064f806b None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:07:14.308398 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 680/1354] 10.222.0.42 () {54 vars in 921 bytes} [Mon Jul 25 12:07:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:07:14.314082 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbda372-c9bc-4fca-b480-43e0a2a3ab8f None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:07:14.314484 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbda372-c9bc-4fca-b480-43e0a2a3ab8f None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:07:14.314657 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbda372-c9bc-4fca-b480-43e0a2a3ab8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:07:14.392487 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-6cbda372-c9bc-4fca-b480-43e0a2a3ab8f None None] MFA Rules not processed for user `8012afab6cac4358ab878732e58d0689`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:07:14.393621 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-6cbda372-c9bc-4fca-b480-43e0a2a3ab8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:07:14.498415 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 675/1355] 10.222.0.42 () {58 vars in 1018 bytes} [Mon Jul 25 12:07:14 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 12:07:17.901216 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5906c819-9264-4010-98b5-8cb3a9743f97 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:07:17.902743 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5906c819-9264-4010-98b5-8cb3a9743f97 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:07:17.903285 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5906c819-9264-4010-98b5-8cb3a9743f97 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:07:17.905178 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 681/1356] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 25 12:07:17 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 12:07:17.910726 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb830e2-0c80-4687-af1f-fdb29f9c1629 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:07:17.911160 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb830e2-0c80-4687-af1f-fdb29f9c1629 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:07:17.911292 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb830e2-0c80-4687-af1f-fdb29f9c1629 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:07:17.959009 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:07:17.995431 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-4bb830e2-0c80-4687-af1f-fdb29f9c1629 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:07:17.997190 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-4bb830e2-0c80-4687-af1f-fdb29f9c1629 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:07:18.100115 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 676/1357] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 25 12:07:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 12:07:18.156955 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe89ca9-a092-4a31-b658-b92f8dc4ca74 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:07:18.157333 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe89ca9-a092-4a31-b658-b92f8dc4ca74 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:07:18.157769 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe89ca9-a092-4a31-b658-b92f8dc4ca74 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:07:18.160093 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 682/1358] 10.222.0.42 () {54 vars in 962 bytes} [Mon Jul 25 12:07:18 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 12:07:18.170999 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bdb883-29bf-433c-9b85-a706baf01685 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:07:18.171861 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bdb883-29bf-433c-9b85-a706baf01685 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:07:18.172242 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bdb883-29bf-433c-9b85-a706baf01685 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:07:18.262771 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-f0bdb883-29bf-433c-9b85-a706baf01685 None None] MFA Rules not processed for user `f67142a1193d47c7bd33d7040268233c`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:07:18.264455 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-f0bdb883-29bf-433c-9b85-a706baf01685 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:07:18.317270 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:07:18.391239 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 677/1359] 10.222.0.42 () {58 vars in 1059 bytes} [Mon Jul 25 12:07:18 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:07:18.399576 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe89ca9-a092-4a31-b658-b92f8dc4ca74 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:07:18.401327 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-3fe89ca9-a092-4a31-b658-b92f8dc4ca74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:07:18.556178 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe89ca9-a092-4a31-b658-b92f8dc4ca74 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:07:18.639741 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:07:18.684718 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1785e5-e427-48cd-aa30-00d2f60db3c4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f67142a1193d47c7bd33d7040268233c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:07:18.685289 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1785e5-e427-48cd-aa30-00d2f60db3c4 service nova] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:07:18.685575 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1785e5-e427-48cd-aa30-00d2f60db3c4 service nova] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:07:18.685852 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1785e5-e427-48cd-aa30-00d2f60db3c4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:07:18.687494 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-aa1785e5-e427-48cd-aa30-00d2f60db3c4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:07:18.789114 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1785e5-e427-48cd-aa30-00d2f60db3c4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:07:18.791030 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1785e5-e427-48cd-aa30-00d2f60db3c4 service nova] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:07:18.810192 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 683/1360] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:07:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:07:21.099931 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f19ef3-c2bd-4567-b1c0-07dbfe36612f None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:07:21.099931 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f19ef3-c2bd-4567-b1c0-07dbfe36612f None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:07:21.099931 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f19ef3-c2bd-4567-b1c0-07dbfe36612f None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:07:21.101433 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 678/1361] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 25 12:07:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:07:21.106583 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605e8d0b-ebb3-45a3-a6d7-70ff7315017e None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:07:21.106968 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605e8d0b-ebb3-45a3-a6d7-70ff7315017e None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:07:21.107220 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605e8d0b-ebb3-45a3-a6d7-70ff7315017e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:07:21.177613 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-605e8d0b-ebb3-45a3-a6d7-70ff7315017e None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:07:21.178835 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-605e8d0b-ebb3-45a3-a6d7-70ff7315017e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:07:21.222156 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:07:21.297715 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 684/1362] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 25 12:07:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:07:21.346164 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f19ef3-c2bd-4567-b1c0-07dbfe36612f None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:07:21.347897 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-62f19ef3-c2bd-4567-b1c0-07dbfe36612f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:07:21.519391 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f19ef3-c2bd-4567-b1c0-07dbfe36612f None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:07:21.529916 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:07:21.657620 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e44765d-03d7-4929-9b4a-8114f908c8e7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f67142a1193d47c7bd33d7040268233c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:07:21.658477 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e44765d-03d7-4929-9b4a-8114f908c8e7 service nova] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:07:21.658767 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e44765d-03d7-4929-9b4a-8114f908c8e7 service nova] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:07:21.659049 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e44765d-03d7-4929-9b4a-8114f908c8e7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:07:21.660769 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-4e44765d-03d7-4929-9b4a-8114f908c8e7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:07:21.754838 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e44765d-03d7-4929-9b4a-8114f908c8e7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:07:21.756475 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e44765d-03d7-4929-9b4a-8114f908c8e7 service nova] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:07:21.777987 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 679/1363] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:07:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:07:24.098821 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef3da45-b3ea-49be-8e04-fedeb5c8f814 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:07:24.099855 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef3da45-b3ea-49be-8e04-fedeb5c8f814 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:07:24.100515 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef3da45-b3ea-49be-8e04-fedeb5c8f814 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:07:24.101758 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 685/1364] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 25 12:07:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:07:24.107737 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f117bdf2-8feb-4563-ae00-c0671915b204 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:07:24.108170 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f117bdf2-8feb-4563-ae00-c0671915b204 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:07:24.108324 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f117bdf2-8feb-4563-ae00-c0671915b204 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:07:24.127347 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:07:24.189423 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-f117bdf2-8feb-4563-ae00-c0671915b204 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:07:24.190654 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-f117bdf2-8feb-4563-ae00-c0671915b204 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:07:24.295856 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 680/1365] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 25 12:07:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:07:24.341659 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19e950d-fbdf-477e-9e06-b49882e8d58b None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:07:24.341972 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19e950d-fbdf-477e-9e06-b49882e8d58b None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:07:24.342246 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19e950d-fbdf-477e-9e06-b49882e8d58b None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:07:24.343670 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 686/1366] 10.222.0.42 () {54 vars in 962 bytes} [Mon Jul 25 12:07:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:07:24.349563 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54f96f3-d097-4100-ae40-ead5fa1cd6ea None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:07:24.349888 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54f96f3-d097-4100-ae40-ead5fa1cd6ea None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:07:24.350259 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54f96f3-d097-4100-ae40-ead5fa1cd6ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:07:24.428918 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-e54f96f3-d097-4100-ae40-ead5fa1cd6ea None None] MFA Rules not processed for user `f67142a1193d47c7bd33d7040268233c`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:07:24.430125 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-e54f96f3-d097-4100-ae40-ead5fa1cd6ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:07:24.470027 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:07:24.555582 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 681/1367] 10.222.0.42 () {58 vars in 1059 bytes} [Mon Jul 25 12:07:24 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:07:24.563605 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19e950d-fbdf-477e-9e06-b49882e8d58b None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:07:24.565398 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-c19e950d-fbdf-477e-9e06-b49882e8d58b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:07:24.741176 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19e950d-fbdf-477e-9e06-b49882e8d58b None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:07:24.763380 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:07:24.868362 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c283a54-1ba3-4fa1-98d7-5ead0d2ed196 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f67142a1193d47c7bd33d7040268233c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:07:24.868930 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c283a54-1ba3-4fa1-98d7-5ead0d2ed196 service nova] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:07:24.869215 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c283a54-1ba3-4fa1-98d7-5ead0d2ed196 service nova] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:07:24.869494 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c283a54-1ba3-4fa1-98d7-5ead0d2ed196 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:07:24.871678 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-0c283a54-1ba3-4fa1-98d7-5ead0d2ed196 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:07:24.964901 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c283a54-1ba3-4fa1-98d7-5ead0d2ed196 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:07:24.966544 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c283a54-1ba3-4fa1-98d7-5ead0d2ed196 service nova] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:07:24.985495 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 687/1368] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:07:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:07:26.022363 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca136883-f207-45ee-96bd-7cf451fd3d62 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:07:26.022892 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca136883-f207-45ee-96bd-7cf451fd3d62 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:07:26.023263 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca136883-f207-45ee-96bd-7cf451fd3d62 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:07:26.024870 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 682/1369] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 25 12:07:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:07:26.030378 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a6cc0f-c654-441f-8e9f-efdb8056edcf None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:07:26.030830 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a6cc0f-c654-441f-8e9f-efdb8056edcf None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:07:26.030914 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a6cc0f-c654-441f-8e9f-efdb8056edcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:07:26.064593 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:07:26.100776 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-36a6cc0f-c654-441f-8e9f-efdb8056edcf None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:07:26.102294 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-36a6cc0f-c654-441f-8e9f-efdb8056edcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:07:26.216491 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 688/1370] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 25 12:07:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:07:26.566003 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a549e78-281d-4f0d-ba28-0c0ba6ee9754 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:07:26.566003 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a549e78-281d-4f0d-ba28-0c0ba6ee9754 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:07:26.566003 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a549e78-281d-4f0d-ba28-0c0ba6ee9754 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:07:26.567008 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 683/1371] 10.222.0.42 () {54 vars in 964 bytes} [Mon Jul 25 12:07:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:07:26.571597 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4136e708-86c5-45b7-923f-5ae72c6517d2 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:07:26.571896 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4136e708-86c5-45b7-923f-5ae72c6517d2 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:07:26.572180 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4136e708-86c5-45b7-923f-5ae72c6517d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:07:26.651098 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-4136e708-86c5-45b7-923f-5ae72c6517d2 None None] MFA Rules not processed for user `6f89ce6f0cf04373a414d927fa0f8a7f`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:07:26.652370 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-4136e708-86c5-45b7-923f-5ae72c6517d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:07:26.705593 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:07:26.738903 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 689/1372] 10.222.0.42 () {58 vars in 1061 bytes} [Mon Jul 25 12:07:26 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:07:26.747035 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a549e78-281d-4f0d-ba28-0c0ba6ee9754 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:07:26.748572 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-7a549e78-281d-4f0d-ba28-0c0ba6ee9754 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:07:26.858276 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a549e78-281d-4f0d-ba28-0c0ba6ee9754 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:07:26.897959 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:07:26.932510 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a52d993-02c8-44d9-bc8f-f66e3f71e2dd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f89ce6f0cf04373a414d927fa0f8a7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:07:26.933916 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a52d993-02c8-44d9-bc8f-f66e3f71e2dd service cinder] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:07:26.934270 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a52d993-02c8-44d9-bc8f-f66e3f71e2dd service cinder] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:07:26.934611 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a52d993-02c8-44d9-bc8f-f66e3f71e2dd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:07:26.936451 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-2a52d993-02c8-44d9-bc8f-f66e3f71e2dd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:07:27.056852 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a52d993-02c8-44d9-bc8f-f66e3f71e2dd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:07:27.057776 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a52d993-02c8-44d9-bc8f-f66e3f71e2dd service cinder] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:07:27.076801 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 684/1373] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:07:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:07:27.082113 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4c3a55-bb02-4aaa-b098-c4fbb7bb728e None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:07:27.082579 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4c3a55-bb02-4aaa-b098-c4fbb7bb728e None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:07:27.082768 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4c3a55-bb02-4aaa-b098-c4fbb7bb728e None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:07:27.084221 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 690/1374] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 25 12:07:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:07:27.089952 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bdc159-9e26-4045-b8fb-59ae3a4dc06a None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:07:27.090245 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bdc159-9e26-4045-b8fb-59ae3a4dc06a None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:07:27.090528 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bdc159-9e26-4045-b8fb-59ae3a4dc06a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:07:27.160121 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-d3bdc159-9e26-4045-b8fb-59ae3a4dc06a None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:07:27.161315 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-d3bdc159-9e26-4045-b8fb-59ae3a4dc06a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:07:27.182427 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:07:27.272060 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 685/1375] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 25 12:07:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:07:27.318156 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4c3a55-bb02-4aaa-b098-c4fbb7bb728e None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:07:27.320305 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-5a4c3a55-bb02-4aaa-b098-c4fbb7bb728e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:07:27.503704 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4c3a55-bb02-4aaa-b098-c4fbb7bb728e None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:07:27.559727 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:07:27.622961 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af329b2-3438-447a-95aa-11cef95ebbff service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f67142a1193d47c7bd33d7040268233c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:07:27.623740 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af329b2-3438-447a-95aa-11cef95ebbff service nova] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:07:27.624119 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af329b2-3438-447a-95aa-11cef95ebbff service nova] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:07:27.624468 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af329b2-3438-447a-95aa-11cef95ebbff service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:07:27.626196 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-3af329b2-3438-447a-95aa-11cef95ebbff service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:07:27.716308 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af329b2-3438-447a-95aa-11cef95ebbff service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:07:27.717960 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af329b2-3438-447a-95aa-11cef95ebbff service nova] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:07:27.736955 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 691/1376] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:07:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:07:29.786544 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c1ee74-2b97-47ee-aca0-c8ee8f002c39 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:07:29.786980 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c1ee74-2b97-47ee-aca0-c8ee8f002c39 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:07:29.787117 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c1ee74-2b97-47ee-aca0-c8ee8f002c39 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:07:29.788749 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 686/1377] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 25 12:07:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:07:29.794377 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89364df3-ada8-474a-b36e-a932439a4882 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:07:29.794804 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89364df3-ada8-474a-b36e-a932439a4882 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:07:29.794941 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89364df3-ada8-474a-b36e-a932439a4882 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:07:29.869842 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-89364df3-ada8-474a-b36e-a932439a4882 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:07:29.871933 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-89364df3-ada8-474a-b36e-a932439a4882 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:07:29.879923 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:07:29.978904 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 692/1378] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 25 12:07:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:07:30.034735 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c1ee74-2b97-47ee-aca0-c8ee8f002c39 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:07:30.036576 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-00c1ee74-2b97-47ee-aca0-c8ee8f002c39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:07:30.190670 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c1ee74-2b97-47ee-aca0-c8ee8f002c39 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:07:30.194617 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:07:30.313177 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58665097-9477-4685-8d92-73146ffd6543 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f67142a1193d47c7bd33d7040268233c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:07:30.313841 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58665097-9477-4685-8d92-73146ffd6543 service nova] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:07:30.314140 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58665097-9477-4685-8d92-73146ffd6543 service nova] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:07:30.314414 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58665097-9477-4685-8d92-73146ffd6543 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:07:30.316102 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-58665097-9477-4685-8d92-73146ffd6543 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:07:30.416330 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58665097-9477-4685-8d92-73146ffd6543 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:07:30.418052 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58665097-9477-4685-8d92-73146ffd6543 service nova] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:07:30.460074 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 687/1379] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:07:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:07:32.527685 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4efa19-0362-4d40-847f-80e02538541d None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:07:32.528238 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4efa19-0362-4d40-847f-80e02538541d None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:07:32.528238 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4efa19-0362-4d40-847f-80e02538541d None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:07:32.530108 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 693/1380] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 25 12:07:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:07:32.536101 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1793cb-dff8-4b6b-b20c-fd9ace3dd2c2 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:07:32.536686 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1793cb-dff8-4b6b-b20c-fd9ace3dd2c2 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:07:32.536686 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1793cb-dff8-4b6b-b20c-fd9ace3dd2c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:07:32.550638 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:07:32.612118 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-ad1793cb-dff8-4b6b-b20c-fd9ace3dd2c2 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:07:32.613348 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-ad1793cb-dff8-4b6b-b20c-fd9ace3dd2c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:07:32.719634 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 688/1381] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 25 12:07:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:07:32.768801 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4efa19-0362-4d40-847f-80e02538541d None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:07:32.770324 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-9f4efa19-0362-4d40-847f-80e02538541d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:07:32.925720 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:07:32.938408 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4efa19-0362-4d40-847f-80e02538541d None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:07:33.059273 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b668552f-8294-483d-8162-56ed52b97845 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f67142a1193d47c7bd33d7040268233c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:07:33.059966 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b668552f-8294-483d-8162-56ed52b97845 service nova] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:07:33.060257 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b668552f-8294-483d-8162-56ed52b97845 service nova] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:07:33.060547 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b668552f-8294-483d-8162-56ed52b97845 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:07:33.062203 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-b668552f-8294-483d-8162-56ed52b97845 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:07:33.158675 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b668552f-8294-483d-8162-56ed52b97845 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:07:33.160330 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b668552f-8294-483d-8162-56ed52b97845 service nova] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:07:33.172008 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:07:33.182379 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 694/1382] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:07:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:07:35.479841 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3c16bc-ec32-4683-9266-f80d31e5832a None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:07:35.480381 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3c16bc-ec32-4683-9266-f80d31e5832a None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:07:35.480477 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3c16bc-ec32-4683-9266-f80d31e5832a None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:07:35.481834 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 689/1383] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 25 12:07:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:07:35.488201 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e548a263-fc58-430f-aa60-ec01e39b5f7f None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:07:35.488590 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e548a263-fc58-430f-aa60-ec01e39b5f7f None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:07:35.488770 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e548a263-fc58-430f-aa60-ec01e39b5f7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:07:35.565306 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-e548a263-fc58-430f-aa60-ec01e39b5f7f None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:07:35.567209 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-e548a263-fc58-430f-aa60-ec01e39b5f7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:07:35.672482 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 695/1384] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 25 12:07:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:07:35.724834 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3c16bc-ec32-4683-9266-f80d31e5832a None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:07:35.726382 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-9d3c16bc-ec32-4683-9266-f80d31e5832a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:07:35.819021 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:07:35.886770 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3c16bc-ec32-4683-9266-f80d31e5832a None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:07:36.024162 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5500ae-2dfd-44af-81a6-059ca7745fd0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f67142a1193d47c7bd33d7040268233c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:07:36.024822 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5500ae-2dfd-44af-81a6-059ca7745fd0 service nova] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:07:36.025115 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5500ae-2dfd-44af-81a6-059ca7745fd0 service nova] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:07:36.025468 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5500ae-2dfd-44af-81a6-059ca7745fd0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:07:36.027195 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-9d5500ae-2dfd-44af-81a6-059ca7745fd0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:07:36.082202 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:07:36.128476 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5500ae-2dfd-44af-81a6-059ca7745fd0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:07:36.130111 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5500ae-2dfd-44af-81a6-059ca7745fd0 service nova] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:07:36.148849 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 690/1385] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:07:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:07:38.150966 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19611b1c-b72e-4fb6-9082-0e8ff4665264 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:07:38.151492 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19611b1c-b72e-4fb6-9082-0e8ff4665264 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:07:38.151793 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19611b1c-b72e-4fb6-9082-0e8ff4665264 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:07:38.153867 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 696/1386] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 25 12:07:38 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:07:38.160821 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8785f0d-8707-4ae0-b4f4-b7246081738e None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:07:38.161247 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8785f0d-8707-4ae0-b4f4-b7246081738e None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:07:38.161557 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8785f0d-8707-4ae0-b4f4-b7246081738e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:07:38.245692 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-d8785f0d-8707-4ae0-b4f4-b7246081738e None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:07:38.247297 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-d8785f0d-8707-4ae0-b4f4-b7246081738e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:07:38.355502 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 691/1387] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 25 12:07:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:07:38.409150 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19611b1c-b72e-4fb6-9082-0e8ff4665264 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:07:38.410731 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-19611b1c-b72e-4fb6-9082-0e8ff4665264 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:07:38.496619 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:07:38.573388 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19611b1c-b72e-4fb6-9082-0e8ff4665264 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:07:38.687898 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b663c84f-e924-4b40-81e2-f5a5bd44d792 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f67142a1193d47c7bd33d7040268233c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:07:38.688472 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b663c84f-e924-4b40-81e2-f5a5bd44d792 service nova] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:07:38.688764 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b663c84f-e924-4b40-81e2-f5a5bd44d792 service nova] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:07:38.689044 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b663c84f-e924-4b40-81e2-f5a5bd44d792 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:07:38.690669 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-b663c84f-e924-4b40-81e2-f5a5bd44d792 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:07:38.730507 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:07:38.791403 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b663c84f-e924-4b40-81e2-f5a5bd44d792 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:07:38.793083 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b663c84f-e924-4b40-81e2-f5a5bd44d792 service nova] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:07:38.811142 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 697/1388] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:07:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:07:40.840674 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3dffac-8c83-4371-8c88-159537d1dce2 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:07:40.841230 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3dffac-8c83-4371-8c88-159537d1dce2 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:07:40.841230 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3dffac-8c83-4371-8c88-159537d1dce2 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:07:40.842778 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 692/1389] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 25 12:07:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:07:40.847913 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdf05e1-1784-4489-8dcf-f20dd2daf06e None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:07:40.848372 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdf05e1-1784-4489-8dcf-f20dd2daf06e None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:07:40.848472 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdf05e1-1784-4489-8dcf-f20dd2daf06e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:07:40.915786 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-7fdf05e1-1784-4489-8dcf-f20dd2daf06e None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:07:40.917105 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-7fdf05e1-1784-4489-8dcf-f20dd2daf06e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:07:41.013691 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 698/1390] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 25 12:07:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:07:41.058686 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3dffac-8c83-4371-8c88-159537d1dce2 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:07:41.060247 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-8d3dffac-8c83-4371-8c88-159537d1dce2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:07:41.092182 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:07:41.210305 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3dffac-8c83-4371-8c88-159537d1dce2 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:07:41.313916 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:07:41.327080 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8ec63c-a2e1-41d6-8cac-07163bfa78c3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f67142a1193d47c7bd33d7040268233c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:07:41.327766 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8ec63c-a2e1-41d6-8cac-07163bfa78c3 service nova] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:07:41.328059 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8ec63c-a2e1-41d6-8cac-07163bfa78c3 service nova] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:07:41.328346 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8ec63c-a2e1-41d6-8cac-07163bfa78c3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:07:41.330018 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-7d8ec63c-a2e1-41d6-8cac-07163bfa78c3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:07:41.421796 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8ec63c-a2e1-41d6-8cac-07163bfa78c3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:07:41.423411 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8ec63c-a2e1-41d6-8cac-07163bfa78c3 service nova] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:07:41.441975 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 693/1391] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:07:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:07:42.355043 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ba45e7-a297-4ef0-866c-6cd488591b4c None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:07:42.355639 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ba45e7-a297-4ef0-866c-6cd488591b4c None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:07:42.355639 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ba45e7-a297-4ef0-866c-6cd488591b4c None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:07:42.357002 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 699/1392] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 25 12:07:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:07:42.362160 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b3e634-7bf6-4b22-a662-d88129f326c1 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:07:42.362569 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b3e634-7bf6-4b22-a662-d88129f326c1 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:07:42.362660 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b3e634-7bf6-4b22-a662-d88129f326c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:07:42.430388 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-58b3e634-7bf6-4b22-a662-d88129f326c1 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:07:42.431667 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-58b3e634-7bf6-4b22-a662-d88129f326c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:07:42.489979 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:07:42.528857 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 694/1393] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 25 12:07:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:07:42.584419 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3e270a-349f-4984-8c5b-90aefe67a8eb None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:07:42.584708 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3e270a-349f-4984-8c5b-90aefe67a8eb None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:07:42.584991 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3e270a-349f-4984-8c5b-90aefe67a8eb None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:07:42.586363 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 700/1394] 10.222.0.42 () {54 vars in 966 bytes} [Mon Jul 25 12:07:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:07:42.591879 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fd3ea7-1884-4455-a4ca-39051fe1e2aa None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:07:42.592162 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fd3ea7-1884-4455-a4ca-39051fe1e2aa None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:07:42.592444 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fd3ea7-1884-4455-a4ca-39051fe1e2aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:07:42.667435 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-f1fd3ea7-1884-4455-a4ca-39051fe1e2aa None None] MFA Rules not processed for user `5ea16528b1374bae9cf54603f7907bfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:07:42.668605 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-f1fd3ea7-1884-4455-a4ca-39051fe1e2aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:07:42.742002 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 695/1395] 10.222.0.42 () {58 vars in 1063 bytes} [Mon Jul 25 12:07:42 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:07:42.749418 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3e270a-349f-4984-8c5b-90aefe67a8eb None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:07:42.750919 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-0d3e270a-349f-4984-8c5b-90aefe67a8eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:07:42.765359 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:07:42.856614 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3e270a-349f-4984-8c5b-90aefe67a8eb None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:07:42.929026 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7224f3f3-3583-4a64-b622-81ee5d442e46 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ea16528b1374bae9cf54603f7907bfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:07:42.929623 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7224f3f3-3583-4a64-b622-81ee5d442e46 service glance] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:07:42.929913 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7224f3f3-3583-4a64-b622-81ee5d442e46 service glance] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:07:42.930197 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7224f3f3-3583-4a64-b622-81ee5d442e46 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:07:42.931900 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-7224f3f3-3583-4a64-b622-81ee5d442e46 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:07:43.000134 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:07:43.041689 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7224f3f3-3583-4a64-b622-81ee5d442e46 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:07:43.043127 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7224f3f3-3583-4a64-b622-81ee5d442e46 service glance] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:07:43.061208 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 701/1396] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:07:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:07:43.487606 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b36d05-689b-4c45-96ca-bc53f5f767f6 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:07:43.488139 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b36d05-689b-4c45-96ca-bc53f5f767f6 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:07:43.488139 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b36d05-689b-4c45-96ca-bc53f5f767f6 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:07:43.489531 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 696/1397] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 25 12:07:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:07:43.495143 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4462e33a-f65f-43d2-bdc5-b37a4ab493eb None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:07:43.495618 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4462e33a-f65f-43d2-bdc5-b37a4ab493eb None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:07:43.495729 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4462e33a-f65f-43d2-bdc5-b37a4ab493eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:07:43.563292 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-4462e33a-f65f-43d2-bdc5-b37a4ab493eb None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:07:43.564547 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-4462e33a-f65f-43d2-bdc5-b37a4ab493eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:07:43.666798 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 702/1398] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 25 12:07:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:07:43.713949 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b36d05-689b-4c45-96ca-bc53f5f767f6 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:07:43.715422 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-94b36d05-689b-4c45-96ca-bc53f5f767f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:07:43.782202 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:07:43.821206 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb7d7a1-cda8-4af3-88b7-8afb6977c668 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:07:43.821498 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb7d7a1-cda8-4af3-88b7-8afb6977c668 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:07:43.821832 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb7d7a1-cda8-4af3-88b7-8afb6977c668 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:07:43.848029 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:07:43.872393 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b36d05-689b-4c45-96ca-bc53f5f767f6 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:07:43.889212 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-dcb7d7a1-cda8-4af3-88b7-8afb6977c668 None None] MFA Rules not processed for user `93207b858d294d049fa56d45810a40aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:07:43.890446 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-dcb7d7a1-cda8-4af3-88b7-8afb6977c668 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:07:43.977789 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 703/1399] 10.222.0.42 () {58 vars in 1009 bytes} [Mon Jul 25 12:07:43 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 12:07:43.986081 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c1230b-4d46-4f01-99f7-d6cb2e31d1da service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f67142a1193d47c7bd33d7040268233c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:07:43.986741 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c1230b-4d46-4f01-99f7-d6cb2e31d1da service nova] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:07:43.987420 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c1230b-4d46-4f01-99f7-d6cb2e31d1da service nova] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:07:43.987420 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c1230b-4d46-4f01-99f7-d6cb2e31d1da service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:07:43.988986 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-c6c1230b-4d46-4f01-99f7-d6cb2e31d1da service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:07:44.012272 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:07:44.027045 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4433687f-d1a1-4cd8-b0fc-20b86aa733b9 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:07:44.027343 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4433687f-d1a1-4cd8-b0fc-20b86aa733b9 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:07:44.027664 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4433687f-d1a1-4cd8-b0fc-20b86aa733b9 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:07:44.029470 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 704/1400] 10.222.0.42 () {54 vars in 943 bytes} [Mon Jul 25 12:07:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 12:07:44.035635 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b25b5cc-b1cc-48e7-b1ba-072b46ec886e None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:07:44.035932 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b25b5cc-b1cc-48e7-b1ba-072b46ec886e None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:07:44.036218 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b25b5cc-b1cc-48e7-b1ba-072b46ec886e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:07:44.083810 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c1230b-4d46-4f01-99f7-d6cb2e31d1da service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:07:44.085615 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c1230b-4d46-4f01-99f7-d6cb2e31d1da service nova] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:07:44.103816 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 697/1401] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:07:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:07:44.114893 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-9b25b5cc-b1cc-48e7-b1ba-072b46ec886e None None] MFA Rules not processed for user `7bd782ea66364eeb96592096eae63d4f`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:07:44.116207 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-9b25b5cc-b1cc-48e7-b1ba-072b46ec886e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:07:44.176381 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:07:44.239703 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 705/1402] 10.222.0.42 () {58 vars in 1040 bytes} [Mon Jul 25 12:07:44 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 12:07:44.250169 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c1230b-4d46-4f01-99f7-d6cb2e31d1da service nova] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:07:44.251463 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-c6c1230b-4d46-4f01-99f7-d6cb2e31d1da service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:07:44.347350 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c1230b-4d46-4f01-99f7-d6cb2e31d1da service nova] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:07:44.404943 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:07:44.452254 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12629805-1a12-418d-a753-87dadf1a4d63 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bd782ea66364eeb96592096eae63d4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:07:44.452914 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12629805-1a12-418d-a753-87dadf1a4d63 service swift] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:07:44.453202 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12629805-1a12-418d-a753-87dadf1a4d63 service swift] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:07:44.453487 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12629805-1a12-418d-a753-87dadf1a4d63 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:07:44.455221 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-12629805-1a12-418d-a753-87dadf1a4d63 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:07:44.537781 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12629805-1a12-418d-a753-87dadf1a4d63 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:07:44.539305 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12629805-1a12-418d-a753-87dadf1a4d63 service swift] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:07:44.548066 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 698/1403] 10.222.0.42 () {60 vars in 1384 bytes} [Mon Jul 25 12:07:44 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 25 12:07:46.179603 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92608d2-4852-481d-9ecc-bb076edf1941 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:07:46.180197 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92608d2-4852-481d-9ecc-bb076edf1941 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:07:46.180197 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92608d2-4852-481d-9ecc-bb076edf1941 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:07:46.181707 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 706/1404] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 25 12:07:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:07:46.186700 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6245ddbe-e541-4144-8a9a-46454b77de3c None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:07:46.187196 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6245ddbe-e541-4144-8a9a-46454b77de3c None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:07:46.187285 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6245ddbe-e541-4144-8a9a-46454b77de3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:07:46.262509 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-6245ddbe-e541-4144-8a9a-46454b77de3c None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:07:46.264010 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-6245ddbe-e541-4144-8a9a-46454b77de3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:07:46.308447 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:07:46.388098 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 699/1405] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 25 12:07:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:07:46.422022 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92608d2-4852-481d-9ecc-bb076edf1941 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:07:46.423773 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-a92608d2-4852-481d-9ecc-bb076edf1941 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:07:46.593657 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92608d2-4852-481d-9ecc-bb076edf1941 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:07:46.622293 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:07:46.721643 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f87af12-6eaa-40c8-82ff-ef7e52d8d7a4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f67142a1193d47c7bd33d7040268233c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:07:46.722208 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f87af12-6eaa-40c8-82ff-ef7e52d8d7a4 service nova] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:07:46.722494 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f87af12-6eaa-40c8-82ff-ef7e52d8d7a4 service nova] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:07:46.722775 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f87af12-6eaa-40c8-82ff-ef7e52d8d7a4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:07:46.724569 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-5f87af12-6eaa-40c8-82ff-ef7e52d8d7a4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:07:46.832347 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f87af12-6eaa-40c8-82ff-ef7e52d8d7a4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:07:46.833933 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f87af12-6eaa-40c8-82ff-ef7e52d8d7a4 service nova] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:07:46.856014 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 707/1406] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:07:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:07:48.892980 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d7be39-df09-424b-8c0b-2ca1c5dcb993 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:07:48.893556 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d7be39-df09-424b-8c0b-2ca1c5dcb993 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:07:48.893556 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d7be39-df09-424b-8c0b-2ca1c5dcb993 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:07:48.895036 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 700/1407] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 25 12:07:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 12:07:48.900635 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c293db-2abb-4236-a885-c3a891ba197a None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:07:48.901106 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c293db-2abb-4236-a885-c3a891ba197a None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:07:48.901210 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c293db-2abb-4236-a885-c3a891ba197a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:07:48.943304 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:07:48.974189 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-e6c293db-2abb-4236-a885-c3a891ba197a None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:07:48.975435 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-e6c293db-2abb-4236-a885-c3a891ba197a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:07:49.083689 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 708/1408] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 25 12:07:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:07:49.128552 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d7be39-df09-424b-8c0b-2ca1c5dcb993 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:07:49.130134 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-86d7be39-df09-424b-8c0b-2ca1c5dcb993 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:07:49.289313 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d7be39-df09-424b-8c0b-2ca1c5dcb993 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:07:49.322427 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:07:49.420284 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f65652-3e76-4363-8517-342c2e6afaa1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f67142a1193d47c7bd33d7040268233c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:07:49.421024 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f65652-3e76-4363-8517-342c2e6afaa1 service nova] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:07:49.421317 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f65652-3e76-4363-8517-342c2e6afaa1 service nova] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:07:49.421598 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f65652-3e76-4363-8517-342c2e6afaa1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:07:49.423677 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-d1f65652-3e76-4363-8517-342c2e6afaa1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:07:49.529111 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f65652-3e76-4363-8517-342c2e6afaa1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:07:49.530720 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f65652-3e76-4363-8517-342c2e6afaa1 service nova] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:07:49.553143 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 701/1409] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:07:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:07:59.081742 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=51 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:07:59.547536 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=86 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:08:06.936026 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbd7461-9581-4078-b9d9-0052fb683930 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:08:06.936586 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbd7461-9581-4078-b9d9-0052fb683930 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:08:06.936586 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbd7461-9581-4078-b9d9-0052fb683930 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:08:06.937971 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 709/1410] 10.222.0.42 () {54 vars in 918 bytes} [Mon Jul 25 12:08:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:08:06.943203 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9ae91a-7722-41a1-a6f8-696d80886c4a None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:08:06.943634 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9ae91a-7722-41a1-a6f8-696d80886c4a None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:08:06.943802 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9ae91a-7722-41a1-a6f8-696d80886c4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:08:07.020422 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-be9ae91a-7722-41a1-a6f8-696d80886c4a None None] MFA Rules not processed for user `8012afab6cac4358ab878732e58d0689`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:08:07.021638 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-be9ae91a-7722-41a1-a6f8-696d80886c4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:08:07.128845 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 702/1411] 10.222.0.42 () {58 vars in 1015 bytes} [Mon Jul 25 12:08:06 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:08:07.145612 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20240a7-493b-404a-9806-651b34634b69 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:08:07.145903 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20240a7-493b-404a-9806-651b34634b69 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:08:07.146187 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20240a7-493b-404a-9806-651b34634b69 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:08:07.147614 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 710/1412] 10.222.0.42 () {54 vars in 918 bytes} [Mon Jul 25 12:08:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:08:07.152748 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ef8dd3-1701-4e61-8bd7-163b5a983495 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:08:07.153044 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ef8dd3-1701-4e61-8bd7-163b5a983495 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:08:07.153331 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ef8dd3-1701-4e61-8bd7-163b5a983495 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:08:07.227785 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:08:07.231102 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-07ef8dd3-1701-4e61-8bd7-163b5a983495 None None] MFA Rules not processed for user `f67142a1193d47c7bd33d7040268233c`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:08:07.233049 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-07ef8dd3-1701-4e61-8bd7-163b5a983495 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:08:07.346762 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 703/1413] 10.222.0.42 () {58 vars in 1015 bytes} [Mon Jul 25 12:08:07 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:08:09.803084 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de29712-840e-43bb-8dbd-7f0df26b2db4 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:08:09.803596 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de29712-840e-43bb-8dbd-7f0df26b2db4 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:08:09.803720 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de29712-840e-43bb-8dbd-7f0df26b2db4 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:08:09.805046 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 711/1414] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 25 12:08:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:08:09.810172 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43405ca6-88bc-49ea-83ab-764eee6a32b3 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:08:09.810618 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43405ca6-88bc-49ea-83ab-764eee6a32b3 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:08:09.810738 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43405ca6-88bc-49ea-83ab-764eee6a32b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:08:09.877923 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-43405ca6-88bc-49ea-83ab-764eee6a32b3 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:08:09.879187 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-43405ca6-88bc-49ea-83ab-764eee6a32b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:08:09.934280 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:08:09.975866 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 704/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 25 12:08:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:08:10.018698 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de29712-840e-43bb-8dbd-7f0df26b2db4 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:08:10.020251 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-3de29712-840e-43bb-8dbd-7f0df26b2db4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:08:10.166351 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de29712-840e-43bb-8dbd-7f0df26b2db4 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:08:10.237279 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:08:10.277523 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12543406-17a0-42bb-bc7f-3785afecb476 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f67142a1193d47c7bd33d7040268233c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:08:10.278170 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12543406-17a0-42bb-bc7f-3785afecb476 service nova] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:08:10.278459 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12543406-17a0-42bb-bc7f-3785afecb476 service nova] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:08:10.278745 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12543406-17a0-42bb-bc7f-3785afecb476 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:08:10.280414 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-12543406-17a0-42bb-bc7f-3785afecb476 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:08:10.370437 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12543406-17a0-42bb-bc7f-3785afecb476 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:08:10.372075 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12543406-17a0-42bb-bc7f-3785afecb476 service nova] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:08:10.389904 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 712/1416] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:08:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:08:11.280995 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec374692-c540-4299-ae76-8c5b4a9ebe8d None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:08:11.281543 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec374692-c540-4299-ae76-8c5b4a9ebe8d None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:08:11.281543 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec374692-c540-4299-ae76-8c5b4a9ebe8d None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:08:11.282964 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 705/1417] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 25 12:08:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:08:11.287853 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2be4ddc-07f3-47f3-ae9c-1aab5090c44c None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:08:11.288296 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2be4ddc-07f3-47f3-ae9c-1aab5090c44c None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:08:11.288423 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2be4ddc-07f3-47f3-ae9c-1aab5090c44c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:08:11.364731 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-a2be4ddc-07f3-47f3-ae9c-1aab5090c44c None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:08:11.365955 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-a2be4ddc-07f3-47f3-ae9c-1aab5090c44c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:08:11.393731 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:08:11.463219 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 713/1418] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 25 12:08:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:08:11.612086 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec374692-c540-4299-ae76-8c5b4a9ebe8d None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:08:11.613613 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-ec374692-c540-4299-ae76-8c5b4a9ebe8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:08:11.761394 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec374692-c540-4299-ae76-8c5b4a9ebe8d None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:08:11.801752 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:08:11.874984 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e72fa0-3e21-478b-a797-788b68b42fa5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f67142a1193d47c7bd33d7040268233c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:08:11.875576 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e72fa0-3e21-478b-a797-788b68b42fa5 service nova] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:08:11.875864 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e72fa0-3e21-478b-a797-788b68b42fa5 service nova] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:08:11.876150 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e72fa0-3e21-478b-a797-788b68b42fa5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:08:11.877786 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-a1e72fa0-3e21-478b-a797-788b68b42fa5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:08:11.964660 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e72fa0-3e21-478b-a797-788b68b42fa5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:08:11.966241 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e72fa0-3e21-478b-a797-788b68b42fa5 service nova] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:08:11.984170 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 706/1419] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:08:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:08:12.862578 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18824bc-81f5-48f7-b35b-3a26dcec9357 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:08:12.863124 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18824bc-81f5-48f7-b35b-3a26dcec9357 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:08:12.863124 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18824bc-81f5-48f7-b35b-3a26dcec9357 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:08:12.864532 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 714/1420] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 25 12:08:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:08:12.870209 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00922d89-e626-4653-a680-4481576f75ae None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:08:12.870623 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00922d89-e626-4653-a680-4481576f75ae None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:08:12.870775 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00922d89-e626-4653-a680-4481576f75ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:08:12.927119 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:08:12.951129 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-00922d89-e626-4653-a680-4481576f75ae None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:08:12.952392 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-00922d89-e626-4653-a680-4481576f75ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:08:13.048035 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 707/1421] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 25 12:08:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:08:13.199979 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18824bc-81f5-48f7-b35b-3a26dcec9357 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:08:13.201830 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-f18824bc-81f5-48f7-b35b-3a26dcec9357 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:08:13.348959 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18824bc-81f5-48f7-b35b-3a26dcec9357 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:08:13.361057 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:08:13.463070 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ddf8b7-6d95-4946-a954-b8226c543207 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f67142a1193d47c7bd33d7040268233c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:08:13.463751 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ddf8b7-6d95-4946-a954-b8226c543207 service nova] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:08:13.464045 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ddf8b7-6d95-4946-a954-b8226c543207 service nova] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:08:13.464332 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ddf8b7-6d95-4946-a954-b8226c543207 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:08:13.465973 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-b9ddf8b7-6d95-4946-a954-b8226c543207 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:08:13.555430 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ddf8b7-6d95-4946-a954-b8226c543207 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:08:13.557055 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ddf8b7-6d95-4946-a954-b8226c543207 service nova] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:08:13.575223 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 715/1422] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:08:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:08:14.439222 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919dff32-6215-41e9-b9c3-3236a69e7b3c None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:08:14.439794 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919dff32-6215-41e9-b9c3-3236a69e7b3c None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:08:14.439794 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919dff32-6215-41e9-b9c3-3236a69e7b3c None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:08:14.441093 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 708/1423] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 25 12:08:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:08:14.446722 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e84ca30-c754-41d6-80f1-25689d5465ed None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:08:14.447173 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e84ca30-c754-41d6-80f1-25689d5465ed None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:08:14.447684 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e84ca30-c754-41d6-80f1-25689d5465ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:08:14.468505 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:08:14.528875 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-6e84ca30-c754-41d6-80f1-25689d5465ed None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:08:14.530098 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-6e84ca30-c754-41d6-80f1-25689d5465ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:08:14.625054 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 716/1424] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 25 12:08:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:08:14.776147 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919dff32-6215-41e9-b9c3-3236a69e7b3c None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:08:14.777858 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-919dff32-6215-41e9-b9c3-3236a69e7b3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:08:14.887678 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:08:14.923901 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919dff32-6215-41e9-b9c3-3236a69e7b3c None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:08:15.035782 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e635a5-48f6-43c1-98ec-aa4e1e98daa9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f67142a1193d47c7bd33d7040268233c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:08:15.036364 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e635a5-48f6-43c1-98ec-aa4e1e98daa9 service nova] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:08:15.036656 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e635a5-48f6-43c1-98ec-aa4e1e98daa9 service nova] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:08:15.036941 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e635a5-48f6-43c1-98ec-aa4e1e98daa9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:08:15.039224 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-d1e635a5-48f6-43c1-98ec-aa4e1e98daa9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:08:15.113340 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:08:15.127106 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e635a5-48f6-43c1-98ec-aa4e1e98daa9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:08:15.128758 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e635a5-48f6-43c1-98ec-aa4e1e98daa9 service nova] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:08:15.148377 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 709/1425] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:08:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:08:16.017712 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af578202-6544-41c8-82e7-6f627e87283d None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:08:16.018269 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af578202-6544-41c8-82e7-6f627e87283d None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:08:16.018269 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af578202-6544-41c8-82e7-6f627e87283d None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:08:16.019688 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 717/1426] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 25 12:08:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:08:16.024695 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d91f815-b796-463f-b9bf-1c0377c9ab9b None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:08:16.025080 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d91f815-b796-463f-b9bf-1c0377c9ab9b None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:08:16.025269 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d91f815-b796-463f-b9bf-1c0377c9ab9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:08:16.100595 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-2d91f815-b796-463f-b9bf-1c0377c9ab9b None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:08:16.101834 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-2d91f815-b796-463f-b9bf-1c0377c9ab9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:08:16.201780 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 710/1427] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 25 12:08:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:08:16.356106 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af578202-6544-41c8-82e7-6f627e87283d None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:08:16.357551 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-af578202-6544-41c8-82e7-6f627e87283d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:08:16.441346 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:08:16.502976 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af578202-6544-41c8-82e7-6f627e87283d None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:08:16.616489 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4391b456-cb23-411a-b520-0a4333acfe80 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f67142a1193d47c7bd33d7040268233c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:08:16.617068 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4391b456-cb23-411a-b520-0a4333acfe80 service nova] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:08:16.617358 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4391b456-cb23-411a-b520-0a4333acfe80 service nova] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:08:16.617643 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4391b456-cb23-411a-b520-0a4333acfe80 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:08:16.619273 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-4391b456-cb23-411a-b520-0a4333acfe80 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:08:16.666894 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:08:16.707899 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4391b456-cb23-411a-b520-0a4333acfe80 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:08:16.709544 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4391b456-cb23-411a-b520-0a4333acfe80 service nova] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:08:16.728909 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 718/1428] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:08:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 12:08:17.537806 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baadef7e-cf8c-413f-9091-302e1fd6eae7 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:08:17.539012 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baadef7e-cf8c-413f-9091-302e1fd6eae7 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:08:17.539012 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baadef7e-cf8c-413f-9091-302e1fd6eae7 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:08:17.540349 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 711/1429] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 25 12:08:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:08:17.545487 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d508f65-dc69-4711-933f-358c1e3abd34 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:08:17.545929 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d508f65-dc69-4711-933f-358c1e3abd34 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:08:17.546079 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d508f65-dc69-4711-933f-358c1e3abd34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:08:17.623103 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-7d508f65-dc69-4711-933f-358c1e3abd34 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:08:17.624354 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-7d508f65-dc69-4711-933f-358c1e3abd34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:08:17.720579 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 719/1430] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 25 12:08:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 12:08:17.871097 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baadef7e-cf8c-413f-9091-302e1fd6eae7 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:08:17.872686 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-baadef7e-cf8c-413f-9091-302e1fd6eae7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:08:17.921492 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:08:18.018473 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baadef7e-cf8c-413f-9091-302e1fd6eae7 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:08:18.129977 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e25b7a8-8b43-4195-9c6a-fa050a2462c9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f67142a1193d47c7bd33d7040268233c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:08:18.130553 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e25b7a8-8b43-4195-9c6a-fa050a2462c9 service nova] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:08:18.130845 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e25b7a8-8b43-4195-9c6a-fa050a2462c9 service nova] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:08:18.131132 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e25b7a8-8b43-4195-9c6a-fa050a2462c9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:08:18.132865 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-1e25b7a8-8b43-4195-9c6a-fa050a2462c9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:08:18.139909 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:08:18.221730 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e25b7a8-8b43-4195-9c6a-fa050a2462c9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:08:18.223320 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e25b7a8-8b43-4195-9c6a-fa050a2462c9 service nova] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:08:18.241388 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 712/1431] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:08:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:08:19.051660 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19adee3-408f-4c6c-8a0e-05a236c83d34 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:08:19.052201 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19adee3-408f-4c6c-8a0e-05a236c83d34 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:08:19.052322 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19adee3-408f-4c6c-8a0e-05a236c83d34 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:08:19.054091 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 720/1432] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 25 12:08:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:08:19.058885 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab76bf04-03e1-476d-a542-c926b179b408 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:08:19.059285 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab76bf04-03e1-476d-a542-c926b179b408 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:08:19.059455 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab76bf04-03e1-476d-a542-c926b179b408 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:08:19.134426 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-ab76bf04-03e1-476d-a542-c926b179b408 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:08:19.135655 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-ab76bf04-03e1-476d-a542-c926b179b408 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:08:19.204322 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:08:19.233595 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 713/1433] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 25 12:08:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:08:19.383359 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19adee3-408f-4c6c-8a0e-05a236c83d34 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:08:19.384850 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-d19adee3-408f-4c6c-8a0e-05a236c83d34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:08:19.402700 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:08:19.530104 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19adee3-408f-4c6c-8a0e-05a236c83d34 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:08:19.616601 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:08:19.642299 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7156146-ede1-461c-8d51-829461836bce service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f67142a1193d47c7bd33d7040268233c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:08:19.642900 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7156146-ede1-461c-8d51-829461836bce service nova] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:08:19.643191 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7156146-ede1-461c-8d51-829461836bce service nova] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:08:19.643477 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7156146-ede1-461c-8d51-829461836bce service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:08:19.645149 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-a7156146-ede1-461c-8d51-829461836bce service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:08:19.740392 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7156146-ede1-461c-8d51-829461836bce service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:08:19.742021 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7156146-ede1-461c-8d51-829461836bce service nova] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:08:19.760125 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 721/1434] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:08:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:08:20.588001 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0360d83-5fba-4805-8569-4f27dc9a7cb1 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:08:20.588605 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0360d83-5fba-4805-8569-4f27dc9a7cb1 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:08:20.588605 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0360d83-5fba-4805-8569-4f27dc9a7cb1 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:08:20.589926 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 714/1435] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 25 12:08:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:08:20.596906 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2215fcca-038c-45e8-8628-5e57678bb4b7 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:08:20.597367 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2215fcca-038c-45e8-8628-5e57678bb4b7 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:08:20.597708 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2215fcca-038c-45e8-8628-5e57678bb4b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:08:20.698672 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-2215fcca-038c-45e8-8628-5e57678bb4b7 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:08:20.700007 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-2215fcca-038c-45e8-8628-5e57678bb4b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:08:20.743281 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:08:20.799863 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 722/1436] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 25 12:08:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:08:20.955342 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0360d83-5fba-4805-8569-4f27dc9a7cb1 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:08:20.956824 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-c0360d83-5fba-4805-8569-4f27dc9a7cb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:08:21.100238 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0360d83-5fba-4805-8569-4f27dc9a7cb1 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:08:21.154202 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:08:21.212029 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd584b2-b6cc-43e2-ae34-75ebb6b4a8d3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f67142a1193d47c7bd33d7040268233c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:08:21.212609 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd584b2-b6cc-43e2-ae34-75ebb6b4a8d3 service nova] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:08:21.212895 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd584b2-b6cc-43e2-ae34-75ebb6b4a8d3 service nova] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:08:21.213177 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd584b2-b6cc-43e2-ae34-75ebb6b4a8d3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:08:21.214835 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-0cd584b2-b6cc-43e2-ae34-75ebb6b4a8d3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:08:21.306206 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd584b2-b6cc-43e2-ae34-75ebb6b4a8d3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:08:21.307842 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd584b2-b6cc-43e2-ae34-75ebb6b4a8d3 service nova] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:08:21.326260 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 715/1437] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:08:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:08:22.137223 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4bb07e-7ef7-4d12-b467-51d487e513a1 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:08:22.137875 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4bb07e-7ef7-4d12-b467-51d487e513a1 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:08:22.137875 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4bb07e-7ef7-4d12-b467-51d487e513a1 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:08:22.139228 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 723/1438] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 25 12:08:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:08:22.144352 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e3cdac-57aa-4360-b84d-f48e510073f1 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:08:22.144748 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e3cdac-57aa-4360-b84d-f48e510073f1 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:08:22.144916 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e3cdac-57aa-4360-b84d-f48e510073f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:08:22.220218 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-e9e3cdac-57aa-4360-b84d-f48e510073f1 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:08:22.221438 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-e9e3cdac-57aa-4360-b84d-f48e510073f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:08:22.223491 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:08:22.323440 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 716/1439] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 25 12:08:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:08:22.473358 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4bb07e-7ef7-4d12-b467-51d487e513a1 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:08:22.474816 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-7d4bb07e-7ef7-4d12-b467-51d487e513a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:08:22.623765 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4bb07e-7ef7-4d12-b467-51d487e513a1 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:08:22.650896 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:08:22.737212 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857e654d-f137-4211-b0ab-a5e7e2e1b324 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f67142a1193d47c7bd33d7040268233c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:08:22.737785 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857e654d-f137-4211-b0ab-a5e7e2e1b324 service nova] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:08:22.738072 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857e654d-f137-4211-b0ab-a5e7e2e1b324 service nova] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:08:22.738352 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857e654d-f137-4211-b0ab-a5e7e2e1b324 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:08:22.739999 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-857e654d-f137-4211-b0ab-a5e7e2e1b324 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:08:22.829815 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857e654d-f137-4211-b0ab-a5e7e2e1b324 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:08:22.831439 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857e654d-f137-4211-b0ab-a5e7e2e1b324 service nova] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:08:22.850144 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 724/1440] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:08:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:08:24.748804 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f9e63d-c2fd-4167-ad58-6090558dcf03 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:08:24.749356 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f9e63d-c2fd-4167-ad58-6090558dcf03 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:08:24.749356 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f9e63d-c2fd-4167-ad58-6090558dcf03 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:08:24.750652 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 717/1441] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 25 12:08:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:08:24.755931 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cee5152-4c88-42ed-a358-e11548cb3d12 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:08:24.756335 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cee5152-4c88-42ed-a358-e11548cb3d12 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:08:24.756498 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cee5152-4c88-42ed-a358-e11548cb3d12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:08:24.798765 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:08:24.823721 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-4cee5152-4c88-42ed-a358-e11548cb3d12 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:08:24.824936 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-4cee5152-4c88-42ed-a358-e11548cb3d12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:08:24.925693 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 725/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 25 12:08:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 12:08:24.963968 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f9e63d-c2fd-4167-ad58-6090558dcf03 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:08:24.965510 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-71f9e63d-c2fd-4167-ad58-6090558dcf03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:08:25.068428 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f9e63d-c2fd-4167-ad58-6090558dcf03 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:08:25.100020 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:08:25.141462 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86133c8a-6cd9-4731-9bc2-84410bb7ba5d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51733781201243e8abf9eca39b207750', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:08:25.142112 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86133c8a-6cd9-4731-9bc2-84410bb7ba5d service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:08:25.142404 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86133c8a-6cd9-4731-9bc2-84410bb7ba5d service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:08:25.142694 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86133c8a-6cd9-4731-9bc2-84410bb7ba5d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:08:25.144360 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-86133c8a-6cd9-4731-9bc2-84410bb7ba5d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:08:25.252019 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86133c8a-6cd9-4731-9bc2-84410bb7ba5d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:08:25.253458 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86133c8a-6cd9-4731-9bc2-84410bb7ba5d service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:08:25.272101 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 718/1443] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:08:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:08:27.411045 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663caf86-82ab-49f8-8924-eaa26c73c443 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:08:27.411643 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663caf86-82ab-49f8-8924-eaa26c73c443 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:08:27.411643 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663caf86-82ab-49f8-8924-eaa26c73c443 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:08:27.413065 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 726/1444] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 25 12:08:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:08:27.418514 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a573e1-d254-469f-a89d-048b6942e98f None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:08:27.418928 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a573e1-d254-469f-a89d-048b6942e98f None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:08:27.419088 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a573e1-d254-469f-a89d-048b6942e98f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:08:27.484082 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-f2a573e1-d254-469f-a89d-048b6942e98f None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:08:27.485289 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-f2a573e1-d254-469f-a89d-048b6942e98f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:08:27.497969 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:08:27.583347 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 719/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 25 12:08:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:08:27.621139 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663caf86-82ab-49f8-8924-eaa26c73c443 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:08:27.622744 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-663caf86-82ab-49f8-8924-eaa26c73c443 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:08:27.726949 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663caf86-82ab-49f8-8924-eaa26c73c443 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:08:27.738849 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:08:27.804820 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607c2d97-c864-4b77-a8d5-344494cc6f96 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51733781201243e8abf9eca39b207750', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:08:27.805480 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607c2d97-c864-4b77-a8d5-344494cc6f96 service neutron] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:08:27.805770 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607c2d97-c864-4b77-a8d5-344494cc6f96 service neutron] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:08:27.806058 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607c2d97-c864-4b77-a8d5-344494cc6f96 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:08:27.807734 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-607c2d97-c864-4b77-a8d5-344494cc6f96 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:08:27.915531 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607c2d97-c864-4b77-a8d5-344494cc6f96 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:08:27.917006 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607c2d97-c864-4b77-a8d5-344494cc6f96 service neutron] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:08:27.935325 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 727/1446] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:08:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:08:30.246752 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b540f43f-2958-4d5e-9a9c-a18a2f6e74b3 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:08:30.247296 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b540f43f-2958-4d5e-9a9c-a18a2f6e74b3 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:08:30.247296 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b540f43f-2958-4d5e-9a9c-a18a2f6e74b3 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:08:30.248722 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 720/1447] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 25 12:08:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:08:30.253892 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcc7a1c-de80-4aae-b09d-f78a42fa3215 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:08:30.254304 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcc7a1c-de80-4aae-b09d-f78a42fa3215 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:08:30.254469 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcc7a1c-de80-4aae-b09d-f78a42fa3215 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:08:30.302467 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:08:30.321254 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-4bcc7a1c-de80-4aae-b09d-f78a42fa3215 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:08:30.322480 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-4bcc7a1c-de80-4aae-b09d-f78a42fa3215 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:08:30.417583 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 728/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 25 12:08:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:08:30.456337 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b540f43f-2958-4d5e-9a9c-a18a2f6e74b3 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:08:30.457880 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-b540f43f-2958-4d5e-9a9c-a18a2f6e74b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:08:30.562521 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b540f43f-2958-4d5e-9a9c-a18a2f6e74b3 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:08:30.602492 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:08:30.636346 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0f4706-47fe-4dc6-abea-9978bbd2c526 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51733781201243e8abf9eca39b207750', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:08:30.636999 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0f4706-47fe-4dc6-abea-9978bbd2c526 service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:08:30.637310 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0f4706-47fe-4dc6-abea-9978bbd2c526 service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:08:30.637593 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0f4706-47fe-4dc6-abea-9978bbd2c526 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:08:30.639257 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-0a0f4706-47fe-4dc6-abea-9978bbd2c526 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:08:30.747666 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0f4706-47fe-4dc6-abea-9978bbd2c526 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:08:30.749043 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0f4706-47fe-4dc6-abea-9978bbd2c526 service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:08:30.766725 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 721/1449] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:08:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 12:08:32.893343 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d0bdd2-f261-46a2-b8d1-013f07bfcf2a None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:08:32.893899 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d0bdd2-f261-46a2-b8d1-013f07bfcf2a None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:08:32.893899 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d0bdd2-f261-46a2-b8d1-013f07bfcf2a None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:08:32.895269 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 729/1450] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 25 12:08:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:08:32.901150 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a76cbb1-148c-425f-8b87-79bda91b53ed None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:08:32.901535 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a76cbb1-148c-425f-8b87-79bda91b53ed None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:08:32.901759 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a76cbb1-148c-425f-8b87-79bda91b53ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:08:32.966622 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-2a76cbb1-148c-425f-8b87-79bda91b53ed None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:08:32.967873 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-2a76cbb1-148c-425f-8b87-79bda91b53ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:08:32.988210 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:08:33.064137 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 722/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 25 12:08:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:08:33.102684 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d0bdd2-f261-46a2-b8d1-013f07bfcf2a None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:08:33.104267 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-c5d0bdd2-f261-46a2-b8d1-013f07bfcf2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:08:33.325437 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d0bdd2-f261-46a2-b8d1-013f07bfcf2a None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:08:33.344745 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:08:33.397709 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4deaec4c-6c1e-4a3b-8832-0cc3543cf57e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51733781201243e8abf9eca39b207750', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:08:33.398380 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4deaec4c-6c1e-4a3b-8832-0cc3543cf57e service neutron] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:08:33.398665 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4deaec4c-6c1e-4a3b-8832-0cc3543cf57e service neutron] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:08:33.398943 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4deaec4c-6c1e-4a3b-8832-0cc3543cf57e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:08:33.400668 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-4deaec4c-6c1e-4a3b-8832-0cc3543cf57e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:08:33.508095 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4deaec4c-6c1e-4a3b-8832-0cc3543cf57e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:08:33.509521 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4deaec4c-6c1e-4a3b-8832-0cc3543cf57e service neutron] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:08:33.527722 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 730/1452] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:08:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:08:35.613195 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930017bc-3038-483e-9eca-4a0346e86147 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:08:35.613734 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930017bc-3038-483e-9eca-4a0346e86147 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:08:35.613867 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930017bc-3038-483e-9eca-4a0346e86147 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:08:35.615157 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 723/1453] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 25 12:08:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:08:35.621292 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff75d11-c1e5-4c88-82fe-dede604a91f8 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:08:35.621735 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff75d11-c1e5-4c88-82fe-dede604a91f8 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:08:35.621857 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff75d11-c1e5-4c88-82fe-dede604a91f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:08:35.686621 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:08:35.689156 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-fff75d11-c1e5-4c88-82fe-dede604a91f8 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:08:35.690943 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-fff75d11-c1e5-4c88-82fe-dede604a91f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:08:35.794394 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 731/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 25 12:08:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:08:35.832552 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930017bc-3038-483e-9eca-4a0346e86147 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:08:35.834096 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-930017bc-3038-483e-9eca-4a0346e86147 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:08:35.941347 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930017bc-3038-483e-9eca-4a0346e86147 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:08:36.000501 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:08:36.027651 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88fdd9f-7184-4aa7-907e-0fe51bada09a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51733781201243e8abf9eca39b207750', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:08:36.028307 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88fdd9f-7184-4aa7-907e-0fe51bada09a service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:08:36.028597 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88fdd9f-7184-4aa7-907e-0fe51bada09a service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:08:36.028878 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88fdd9f-7184-4aa7-907e-0fe51bada09a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:08:36.030541 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-a88fdd9f-7184-4aa7-907e-0fe51bada09a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:08:36.138746 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88fdd9f-7184-4aa7-907e-0fe51bada09a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:08:36.140221 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88fdd9f-7184-4aa7-907e-0fe51bada09a service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:08:36.158076 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 724/1455] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:08:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:08:38.218992 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196db846-10ec-49af-88f7-cbe5fade7c13 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:08:38.219484 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196db846-10ec-49af-88f7-cbe5fade7c13 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:08:38.219620 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196db846-10ec-49af-88f7-cbe5fade7c13 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:08:38.221027 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 732/1456] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 25 12:08:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:08:38.226346 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7be6e8-5c68-4059-b402-48298e377fc5 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:08:38.226758 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7be6e8-5c68-4059-b402-48298e377fc5 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:08:38.226916 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7be6e8-5c68-4059-b402-48298e377fc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:08:38.296434 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-2a7be6e8-5c68-4059-b402-48298e377fc5 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:08:38.297659 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-2a7be6e8-5c68-4059-b402-48298e377fc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:08:38.327179 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:08:38.394923 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 725/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 25 12:08:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:08:38.433509 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196db846-10ec-49af-88f7-cbe5fade7c13 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:08:38.435051 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-196db846-10ec-49af-88f7-cbe5fade7c13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:08:38.538970 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196db846-10ec-49af-88f7-cbe5fade7c13 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:08:38.569255 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:08:38.614502 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa9af66-d7e1-4d91-aa0f-8b62fa4c0f80 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51733781201243e8abf9eca39b207750', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:08:38.615213 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa9af66-d7e1-4d91-aa0f-8b62fa4c0f80 service neutron] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:08:38.615514 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa9af66-d7e1-4d91-aa0f-8b62fa4c0f80 service neutron] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:08:38.615829 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa9af66-d7e1-4d91-aa0f-8b62fa4c0f80 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:08:38.617955 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-afa9af66-d7e1-4d91-aa0f-8b62fa4c0f80 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:08:38.726773 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa9af66-d7e1-4d91-aa0f-8b62fa4c0f80 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:08:38.728232 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa9af66-d7e1-4d91-aa0f-8b62fa4c0f80 service neutron] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:08:38.746433 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 733/1458] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:08:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:08:40.792617 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea99f27d-273a-494f-934b-a539e143437e None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:08:40.793097 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea99f27d-273a-494f-934b-a539e143437e None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:08:40.793211 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea99f27d-273a-494f-934b-a539e143437e None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:08:40.794594 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 726/1459] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 25 12:08:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:08:40.799812 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b865c39f-7dbc-423d-a358-8d9c85600bee None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:08:40.800233 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b865c39f-7dbc-423d-a358-8d9c85600bee None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:08:40.800392 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b865c39f-7dbc-423d-a358-8d9c85600bee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:08:40.866000 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-b865c39f-7dbc-423d-a358-8d9c85600bee None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:08:40.867260 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-b865c39f-7dbc-423d-a358-8d9c85600bee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:08:40.875604 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:08:40.964985 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 734/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 25 12:08:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:08:41.001311 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea99f27d-273a-494f-934b-a539e143437e None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:08:41.002896 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-ea99f27d-273a-494f-934b-a539e143437e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:08:41.106028 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea99f27d-273a-494f-934b-a539e143437e None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:08:41.161193 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:08:41.179224 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0870c4-6121-4965-bf72-0ded84dab898 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51733781201243e8abf9eca39b207750', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:08:41.179927 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0870c4-6121-4965-bf72-0ded84dab898 service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:08:41.180222 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0870c4-6121-4965-bf72-0ded84dab898 service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:08:41.180509 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0870c4-6121-4965-bf72-0ded84dab898 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:08:41.182157 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-fd0870c4-6121-4965-bf72-0ded84dab898 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:08:41.289864 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0870c4-6121-4965-bf72-0ded84dab898 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:08:41.291299 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0870c4-6121-4965-bf72-0ded84dab898 service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:08:41.309558 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 727/1461] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:08:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:08:43.347796 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64f903f-ea34-4c41-ab35-b7bd963a06a7 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:08:43.348376 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64f903f-ea34-4c41-ab35-b7bd963a06a7 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:08:43.348376 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64f903f-ea34-4c41-ab35-b7bd963a06a7 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:08:43.349799 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 735/1462] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 25 12:08:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:08:43.354974 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87aa903-c29c-4975-8b29-98f791ed8a1e None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:08:43.355403 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87aa903-c29c-4975-8b29-98f791ed8a1e None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:08:43.355572 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87aa903-c29c-4975-8b29-98f791ed8a1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:08:43.421811 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-d87aa903-c29c-4975-8b29-98f791ed8a1e None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:08:43.423028 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-d87aa903-c29c-4975-8b29-98f791ed8a1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:08:43.461892 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:08:43.519387 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 728/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 25 12:08:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:08:43.556517 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64f903f-ea34-4c41-ab35-b7bd963a06a7 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:08:43.558024 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-b64f903f-ea34-4c41-ab35-b7bd963a06a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:08:43.665170 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64f903f-ea34-4c41-ab35-b7bd963a06a7 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:08:43.703674 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:08:43.739174 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832437fb-2673-4aa9-95ec-b89973de84e8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51733781201243e8abf9eca39b207750', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:08:43.739870 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832437fb-2673-4aa9-95ec-b89973de84e8 service neutron] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:08:43.740161 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832437fb-2673-4aa9-95ec-b89973de84e8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:08:43.740445 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832437fb-2673-4aa9-95ec-b89973de84e8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:08:43.742575 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-832437fb-2673-4aa9-95ec-b89973de84e8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:08:43.853824 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832437fb-2673-4aa9-95ec-b89973de84e8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:08:43.855252 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832437fb-2673-4aa9-95ec-b89973de84e8 service neutron] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:08:43.873570 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 736/1464] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:08:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:08:45.889838 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fee4cf-4f54-49ba-9102-d20347ece832 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:08:45.890399 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fee4cf-4f54-49ba-9102-d20347ece832 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:08:45.890399 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fee4cf-4f54-49ba-9102-d20347ece832 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:08:45.891840 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 729/1465] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 25 12:08:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:08:45.897087 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455b022a-6517-413a-ad0e-a7a40c798bd2 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:08:45.897487 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455b022a-6517-413a-ad0e-a7a40c798bd2 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:08:45.897652 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455b022a-6517-413a-ad0e-a7a40c798bd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:08:45.963073 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-455b022a-6517-413a-ad0e-a7a40c798bd2 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:08:45.964301 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-455b022a-6517-413a-ad0e-a7a40c798bd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:08:45.982332 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:08:46.058774 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 737/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 25 12:08:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:08:46.095005 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fee4cf-4f54-49ba-9102-d20347ece832 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:08:46.096579 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-05fee4cf-4f54-49ba-9102-d20347ece832 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:08:46.199978 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fee4cf-4f54-49ba-9102-d20347ece832 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:08:46.268303 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:08:46.276240 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a941e8-a1a9-467a-8367-42ad3c0c041e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51733781201243e8abf9eca39b207750', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:08:46.276894 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a941e8-a1a9-467a-8367-42ad3c0c041e service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:08:46.277184 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a941e8-a1a9-467a-8367-42ad3c0c041e service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:08:46.277466 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a941e8-a1a9-467a-8367-42ad3c0c041e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:08:46.279380 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-c0a941e8-a1a9-467a-8367-42ad3c0c041e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:08:46.386610 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a941e8-a1a9-467a-8367-42ad3c0c041e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:08:46.388037 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a941e8-a1a9-467a-8367-42ad3c0c041e service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:08:46.406609 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 730/1467] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:08:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:08:48.429189 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39034b54-7220-4a84-ac12-9b1fbc22a745 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:08:48.430334 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39034b54-7220-4a84-ac12-9b1fbc22a745 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:08:48.430334 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39034b54-7220-4a84-ac12-9b1fbc22a745 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:08:48.431658 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 738/1468] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 25 12:08:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:08:48.437152 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344e9b66-5c5c-4433-9a0f-47a89697c7da None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:08:48.437619 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344e9b66-5c5c-4433-9a0f-47a89697c7da None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:08:48.437731 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344e9b66-5c5c-4433-9a0f-47a89697c7da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:08:48.503488 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-344e9b66-5c5c-4433-9a0f-47a89697c7da None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:08:48.504727 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-344e9b66-5c5c-4433-9a0f-47a89697c7da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:08:48.557059 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:08:48.728456 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 731/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 25 12:08:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 292 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:08:48.764984 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39034b54-7220-4a84-ac12-9b1fbc22a745 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:08:48.766505 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-39034b54-7220-4a84-ac12-9b1fbc22a745 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:08:48.872384 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39034b54-7220-4a84-ac12-9b1fbc22a745 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:08:48.919197 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:08:48.944561 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31771ba5-2e30-43a2-a682-23e6914bc531 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51733781201243e8abf9eca39b207750', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:08:48.945212 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31771ba5-2e30-43a2-a682-23e6914bc531 service neutron] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:08:48.945498 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31771ba5-2e30-43a2-a682-23e6914bc531 service neutron] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:08:48.945777 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31771ba5-2e30-43a2-a682-23e6914bc531 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:08:48.947405 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-31771ba5-2e30-43a2-a682-23e6914bc531 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:08:49.054584 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31771ba5-2e30-43a2-a682-23e6914bc531 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:08:49.056134 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31771ba5-2e30-43a2-a682-23e6914bc531 service neutron] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:08:49.074036 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 739/1470] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:08:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:08:58.725952 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=22 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:08:59.072191 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=66 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:09:08.540929 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c13cb5-4168-4886-9b99-d8539eb0464f None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:09:08.541500 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c13cb5-4168-4886-9b99-d8539eb0464f None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:09:08.541500 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c13cb5-4168-4886-9b99-d8539eb0464f None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:09:08.542857 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 732/1471] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 25 12:09:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:09:08.548035 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be70fe5a-b1a9-4e3a-bb13-3ab357656532 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:09:08.548737 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be70fe5a-b1a9-4e3a-bb13-3ab357656532 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:09:08.548737 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be70fe5a-b1a9-4e3a-bb13-3ab357656532 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:09:08.619853 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-be70fe5a-b1a9-4e3a-bb13-3ab357656532 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:09:08.623380 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-be70fe5a-b1a9-4e3a-bb13-3ab357656532 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:09:08.724032 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 740/1472] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 25 12:09:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:09:08.809754 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ddce7f-e143-4a8e-a3b0-1d6cef1ad11e None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:09:08.810048 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ddce7f-e143-4a8e-a3b0-1d6cef1ad11e None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:09:08.810330 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ddce7f-e143-4a8e-a3b0-1d6cef1ad11e None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:09:08.811730 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 733/1473] 10.222.0.42 () {54 vars in 966 bytes} [Mon Jul 25 12:09:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:09:08.816444 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c370f978-742d-4c8d-98a9-e1d1b4323508 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:09:08.816768 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c370f978-742d-4c8d-98a9-e1d1b4323508 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:09:08.817170 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c370f978-742d-4c8d-98a9-e1d1b4323508 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:09:08.891869 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-c370f978-742d-4c8d-98a9-e1d1b4323508 None None] MFA Rules not processed for user `5ea16528b1374bae9cf54603f7907bfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:09:08.893069 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-c370f978-742d-4c8d-98a9-e1d1b4323508 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:09:08.913644 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:09:08.973654 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 741/1474] 10.222.0.42 () {58 vars in 1063 bytes} [Mon Jul 25 12:09:08 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:09:08.981663 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ddce7f-e143-4a8e-a3b0-1d6cef1ad11e None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:09:08.983118 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-f5ddce7f-e143-4a8e-a3b0-1d6cef1ad11e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:09:09.085153 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ddce7f-e143-4a8e-a3b0-1d6cef1ad11e None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:09:09.156892 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1a9c3e-fc6e-403c-824e-5d9437f8222b service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ea16528b1374bae9cf54603f7907bfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:09:09.157527 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1a9c3e-fc6e-403c-824e-5d9437f8222b service glance] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:09:09.157810 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1a9c3e-fc6e-403c-824e-5d9437f8222b service glance] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:09:09.158086 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1a9c3e-fc6e-403c-824e-5d9437f8222b service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:09:09.159729 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-da1a9c3e-fc6e-403c-824e-5d9437f8222b service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:09:09.211011 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:09:09.264580 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1a9c3e-fc6e-403c-824e-5d9437f8222b service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:09:09.265929 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1a9c3e-fc6e-403c-824e-5d9437f8222b service glance] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:09:09.283998 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 734/1475] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:09:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:09:11.352839 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3dee3c-2093-4560-b633-01dac13d6649 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:09:11.353426 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3dee3c-2093-4560-b633-01dac13d6649 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:09:11.353426 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3dee3c-2093-4560-b633-01dac13d6649 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:09:11.354796 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 742/1476] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 25 12:09:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:09:11.359914 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80362fe-c429-4a1d-aa1b-da22c33916e5 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:09:11.360377 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80362fe-c429-4a1d-aa1b-da22c33916e5 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:09:11.360487 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80362fe-c429-4a1d-aa1b-da22c33916e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:09:11.425334 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-e80362fe-c429-4a1d-aa1b-da22c33916e5 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:09:11.426538 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-e80362fe-c429-4a1d-aa1b-da22c33916e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:09:11.522088 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 735/1477] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 25 12:09:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:09:11.545391 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3dee3c-2093-4560-b633-01dac13d6649 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:09:11.546983 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-4e3dee3c-2093-4560-b633-01dac13d6649 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:09:11.671131 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3dee3c-2093-4560-b633-01dac13d6649 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:09:11.720072 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:09:11.764978 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fdd02e-b963-469f-b71d-cee6f5674076 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:09:11.765630 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fdd02e-b963-469f-b71d-cee6f5674076 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:09:11.765914 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fdd02e-b963-469f-b71d-cee6f5674076 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:09:11.766193 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fdd02e-b963-469f-b71d-cee6f5674076 admin admin] PATH_INFO: `/v3/projects` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:09:11.766917 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fdd02e-b963-469f-b71d-cee6f5674076 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:09:11.769120 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: WARNING py.warnings [None req-34fdd02e-b963-469f-b71d-cee6f5674076 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 25 12:09:11.769120 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: warnings.warn(msg) Jul 25 12:09:11.769120 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: Jul 25 12:09:11.770252 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fdd02e-b963-469f-b71d-cee6f5674076 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:09:11.789204 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 743/1478] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Jul 25 12:09:11 2022] GET /identity/v3/projects => generated 2553 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:09:13.821463 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52cc2ce-b7cd-4940-8d06-b5d768b80c7d None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:09:13.821998 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52cc2ce-b7cd-4940-8d06-b5d768b80c7d None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:09:13.822104 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52cc2ce-b7cd-4940-8d06-b5d768b80c7d None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:09:13.823421 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 736/1479] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 25 12:09:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:09:13.828752 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33183671-4efb-4087-9793-d9347c3ffdc6 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:09:13.829149 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33183671-4efb-4087-9793-d9347c3ffdc6 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:09:13.829267 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33183671-4efb-4087-9793-d9347c3ffdc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:09:13.895080 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-33183671-4efb-4087-9793-d9347c3ffdc6 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:09:13.896343 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-33183671-4efb-4087-9793-d9347c3ffdc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:09:13.992781 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 744/1480] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 25 12:09:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:09:14.036655 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52cc2ce-b7cd-4940-8d06-b5d768b80c7d None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:09:14.038244 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-e52cc2ce-b7cd-4940-8d06-b5d768b80c7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:09:14.048626 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:09:14.184627 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52cc2ce-b7cd-4940-8d06-b5d768b80c7d None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:09:14.264216 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:09:14.296691 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e64fb1-b039-4bda-ad8c-22ff7d804447 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f67142a1193d47c7bd33d7040268233c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:09:14.297347 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e64fb1-b039-4bda-ad8c-22ff7d804447 service nova] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:09:14.297635 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e64fb1-b039-4bda-ad8c-22ff7d804447 service nova] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:09:14.297917 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e64fb1-b039-4bda-ad8c-22ff7d804447 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:09:14.299616 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-47e64fb1-b039-4bda-ad8c-22ff7d804447 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:09:14.387119 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e64fb1-b039-4bda-ad8c-22ff7d804447 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:09:14.388726 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e64fb1-b039-4bda-ad8c-22ff7d804447 service nova] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:09:14.406867 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 737/1481] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:09:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:09:16.566197 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92135919-87d1-4e08-8875-16b3b860f747 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:09:16.566735 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92135919-87d1-4e08-8875-16b3b860f747 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:09:16.566835 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92135919-87d1-4e08-8875-16b3b860f747 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:09:16.568205 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 745/1482] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 25 12:09:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:09:16.573250 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cdf050-7643-47b8-a19a-1bd627add5e0 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:09:16.573674 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cdf050-7643-47b8-a19a-1bd627add5e0 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:09:16.573803 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cdf050-7643-47b8-a19a-1bd627add5e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:09:16.637633 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-e0cdf050-7643-47b8-a19a-1bd627add5e0 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:09:16.638814 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-e0cdf050-7643-47b8-a19a-1bd627add5e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:09:16.685349 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:09:16.735490 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 738/1483] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 25 12:09:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:09:16.770558 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92135919-87d1-4e08-8875-16b3b860f747 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:09:16.772175 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-92135919-87d1-4e08-8875-16b3b860f747 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:09:16.796892 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:09:16.875889 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92135919-87d1-4e08-8875-16b3b860f747 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:09:16.953875 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950a13ea-34e8-481f-8d9d-da9854a64967 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51733781201243e8abf9eca39b207750', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:09:16.954537 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950a13ea-34e8-481f-8d9d-da9854a64967 service neutron] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:09:16.954832 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950a13ea-34e8-481f-8d9d-da9854a64967 service neutron] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:09:16.955114 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950a13ea-34e8-481f-8d9d-da9854a64967 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:09:16.956809 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-950a13ea-34e8-481f-8d9d-da9854a64967 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:09:17.032407 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:09:17.067055 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950a13ea-34e8-481f-8d9d-da9854a64967 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:09:17.068529 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950a13ea-34e8-481f-8d9d-da9854a64967 service neutron] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:09:17.087339 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 746/1484] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:09:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:09:19.126773 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44cd7a8-4ee8-42b7-8414-fa5dfdbd984c None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:09:19.127366 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44cd7a8-4ee8-42b7-8414-fa5dfdbd984c None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:09:19.127366 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44cd7a8-4ee8-42b7-8414-fa5dfdbd984c None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:09:19.128741 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 739/1485] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 25 12:09:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:09:19.133963 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef513300-40ab-418e-abfc-2e03989d5724 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:09:19.134398 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef513300-40ab-418e-abfc-2e03989d5724 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:09:19.134506 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef513300-40ab-418e-abfc-2e03989d5724 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:09:19.199001 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-ef513300-40ab-418e-abfc-2e03989d5724 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:09:19.200231 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-ef513300-40ab-418e-abfc-2e03989d5724 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:09:19.295619 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 747/1486] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 25 12:09:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:09:19.333858 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44cd7a8-4ee8-42b7-8414-fa5dfdbd984c None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:09:19.335453 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-e44cd7a8-4ee8-42b7-8414-fa5dfdbd984c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:09:19.437808 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44cd7a8-4ee8-42b7-8414-fa5dfdbd984c None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:09:19.507570 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:09:19.509914 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ac5e60-543a-40c5-b6d4-0d6057aaf09b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51733781201243e8abf9eca39b207750', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:09:19.510746 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ac5e60-543a-40c5-b6d4-0d6057aaf09b service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:09:19.511228 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ac5e60-543a-40c5-b6d4-0d6057aaf09b service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:09:19.511905 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ac5e60-543a-40c5-b6d4-0d6057aaf09b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:09:19.513600 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-49ac5e60-543a-40c5-b6d4-0d6057aaf09b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:09:19.620572 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ac5e60-543a-40c5-b6d4-0d6057aaf09b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:09:19.622004 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ac5e60-543a-40c5-b6d4-0d6057aaf09b service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:09:19.639491 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 740/1487] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:09:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:09:21.874371 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472c6549-abbc-4a9b-a812-6cfd1f414c85 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:09:21.874922 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472c6549-abbc-4a9b-a812-6cfd1f414c85 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:09:21.874922 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472c6549-abbc-4a9b-a812-6cfd1f414c85 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:09:21.876354 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 748/1488] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 25 12:09:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:09:21.882929 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f32169-7606-458b-bc02-9c16712f12f6 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:09:21.883409 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f32169-7606-458b-bc02-9c16712f12f6 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:09:21.883736 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f32169-7606-458b-bc02-9c16712f12f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:09:21.949785 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-88f32169-7606-458b-bc02-9c16712f12f6 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:09:21.950994 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-88f32169-7606-458b-bc02-9c16712f12f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:09:22.013827 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:09:22.061837 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 741/1489] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 25 12:09:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:09:22.102181 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472c6549-abbc-4a9b-a812-6cfd1f414c85 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:09:22.104412 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-472c6549-abbc-4a9b-a812-6cfd1f414c85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:09:22.140208 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:09:22.209805 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472c6549-abbc-4a9b-a812-6cfd1f414c85 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:09:22.282714 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3137adf-7f3b-455b-b8a1-61bc95bfc350 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51733781201243e8abf9eca39b207750', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:09:22.283372 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3137adf-7f3b-455b-b8a1-61bc95bfc350 service neutron] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:09:22.283708 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3137adf-7f3b-455b-b8a1-61bc95bfc350 service neutron] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:09:22.283998 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3137adf-7f3b-455b-b8a1-61bc95bfc350 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:09:22.285638 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-f3137adf-7f3b-455b-b8a1-61bc95bfc350 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:09:22.375622 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:09:22.406178 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3137adf-7f3b-455b-b8a1-61bc95bfc350 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:09:22.407650 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3137adf-7f3b-455b-b8a1-61bc95bfc350 service neutron] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:09:22.426954 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 749/1490] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:09:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:09:24.729730 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53574295-f90a-4bfd-92df-f05678f945da None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:09:24.730255 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53574295-f90a-4bfd-92df-f05678f945da None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:09:24.730370 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53574295-f90a-4bfd-92df-f05678f945da None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:09:24.731720 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 742/1491] 10.222.0.42 () {54 vars in 926 bytes} [Mon Jul 25 12:09:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:09:24.737034 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8beb0e4b-a210-494b-8073-2487c7d2e4bf None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:09:24.737466 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8beb0e4b-a210-494b-8073-2487c7d2e4bf None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:09:24.737598 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8beb0e4b-a210-494b-8073-2487c7d2e4bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:09:24.804683 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-8beb0e4b-a210-494b-8073-2487c7d2e4bf None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:09:24.805907 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-8beb0e4b-a210-494b-8073-2487c7d2e4bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:09:24.901183 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 750/1492] 10.222.0.42 () {58 vars in 1023 bytes} [Mon Jul 25 12:09:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:09:24.940121 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53574295-f90a-4bfd-92df-f05678f945da None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:09:24.941651 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-53574295-f90a-4bfd-92df-f05678f945da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:09:25.042057 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53574295-f90a-4bfd-92df-f05678f945da None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:09:25.115070 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d86aead-bac4-46a1-9238-690f52af7921 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51733781201243e8abf9eca39b207750', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:09:25.115743 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d86aead-bac4-46a1-9238-690f52af7921 service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:09:25.116033 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d86aead-bac4-46a1-9238-690f52af7921 service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:09:25.116314 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d86aead-bac4-46a1-9238-690f52af7921 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:09:25.117946 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-8d86aead-bac4-46a1-9238-690f52af7921 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:09:25.127487 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:09:25.226325 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d86aead-bac4-46a1-9238-690f52af7921 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:09:25.227710 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d86aead-bac4-46a1-9238-690f52af7921 service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:09:25.246064 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 743/1493] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:09:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:09:34.899232 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=82 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:09:35.243275 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=50 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:10:52.642430 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0b6b9a-0834-4cb3-b837-f700714909ed None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:10:52.643008 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0b6b9a-0834-4cb3-b837-f700714909ed None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:10:52.643008 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0b6b9a-0834-4cb3-b837-f700714909ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:10:52.717250 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-5a0b6b9a-0834-4cb3-b837-f700714909ed None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:10:52.718469 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-5a0b6b9a-0834-4cb3-b837-f700714909ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:10:52.813477 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 751/1494] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:10:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:10:52.821415 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec901b5c-64da-44c8-8290-300e0364e926 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:10:52.821907 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec901b5c-64da-44c8-8290-300e0364e926 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:10:52.822026 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec901b5c-64da-44c8-8290-300e0364e926 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:10:52.904861 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-ec901b5c-64da-44c8-8290-300e0364e926 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:10:52.905997 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-ec901b5c-64da-44c8-8290-300e0364e926 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:10:53.002049 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 744/1495] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:10:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:10:53.006709 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0b6b9a-0834-4cb3-b837-f700714909ed None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:10:53.007958 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-5a0b6b9a-0834-4cb3-b837-f700714909ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:10:53.019409 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0b6b9a-0834-4cb3-b837-f700714909ed None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:10:53.081065 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:10:53.115865 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6eedd8f-8430-4572-94f7-c218ecf50dc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:10:53.116525 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6eedd8f-8430-4572-94f7-c218ecf50dc3 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:10:53.116816 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6eedd8f-8430-4572-94f7-c218ecf50dc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:10:53.117096 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6eedd8f-8430-4572-94f7-c218ecf50dc3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:10:53.117819 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6eedd8f-8430-4572-94f7-c218ecf50dc3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:10:53.118132 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6eedd8f-8430-4572-94f7-c218ecf50dc3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:10:53.119149 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: WARNING py.warnings [None req-a6eedd8f-8430-4572-94f7-c218ecf50dc3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 25 12:10:53.119149 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: warnings.warn(msg) Jul 25 12:10:53.119149 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: Jul 25 12:10:53.120663 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6eedd8f-8430-4572-94f7-c218ecf50dc3 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:10:53.131701 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 752/1496] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Jul 25 12:10:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:10:53.139904 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec901b5c-64da-44c8-8290-300e0364e926 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:10:53.149055 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec901b5c-64da-44c8-8290-300e0364e926 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:10:53.210381 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:10:53.250027 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e51da73-7578-4408-bc29-1775fc591bbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:10:53.250685 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e51da73-7578-4408-bc29-1775fc591bbc admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:10:53.250974 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e51da73-7578-4408-bc29-1775fc591bbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:10:53.251250 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e51da73-7578-4408-bc29-1775fc591bbc admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:10:53.251983 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e51da73-7578-4408-bc29-1775fc591bbc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:10:53.253725 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: WARNING py.warnings [None req-1e51da73-7578-4408-bc29-1775fc591bbc admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 25 12:10:53.253725 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: warnings.warn(msg) Jul 25 12:10:53.253725 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: Jul 25 12:10:53.254913 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e51da73-7578-4408-bc29-1775fc591bbc admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:10:53.311438 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 745/1497] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Jul 25 12:10:53 2022] POST /identity/v3/projects => generated 377 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:10:53.315797 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6eedd8f-8430-4572-94f7-c218ecf50dc3 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:10:53.317323 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-a6eedd8f-8430-4572-94f7-c218ecf50dc3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:10:53.352048 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6eedd8f-8430-4572-94f7-c218ecf50dc3 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:10:53.446652 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813ce3fa-2c29-4b24-bd94-fe7194d1ac38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:10:53.447321 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813ce3fa-2c29-4b24-bd94-fe7194d1ac38 admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:10:53.447645 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813ce3fa-2c29-4b24-bd94-fe7194d1ac38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:10:53.447927 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813ce3fa-2c29-4b24-bd94-fe7194d1ac38 admin admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:10:53.448614 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813ce3fa-2c29-4b24-bd94-fe7194d1ac38 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:10:53.449656 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: WARNING py.warnings [None req-813ce3fa-2c29-4b24-bd94-fe7194d1ac38 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 25 12:10:53.449656 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: warnings.warn(msg) Jul 25 12:10:53.449656 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: Jul 25 12:10:53.450866 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813ce3fa-2c29-4b24-bd94-fe7194d1ac38 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:10:53.491912 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 753/1498] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Jul 25 12:10:53 2022] POST /identity/v3/users => generated 289 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:10:53.495778 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e51da73-7578-4408-bc29-1775fc591bbc admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:10:53.497288 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-1e51da73-7578-4408-bc29-1775fc591bbc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:10:53.565197 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e51da73-7578-4408-bc29-1775fc591bbc admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:10:53.650565 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:10:53.662960 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2473382-c956-4366-9407-4f4a15e27368 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:10:53.663663 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2473382-c956-4366-9407-4f4a15e27368 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:10:53.663951 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2473382-c956-4366-9407-4f4a15e27368 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:10:53.664230 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2473382-c956-4366-9407-4f4a15e27368 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:10:53.664966 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2473382-c956-4366-9407-4f4a15e27368 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:10:53.665985 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: WARNING py.warnings [None req-b2473382-c956-4366-9407-4f4a15e27368 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 25 12:10:53.665985 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: warnings.warn(msg) Jul 25 12:10:53.665985 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: Jul 25 12:10:53.666909 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2473382-c956-4366-9407-4f4a15e27368 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:10:53.679129 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 746/1499] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Jul 25 12:10:53 2022] GET /identity/v3/roles => generated 1420 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:10:53.683317 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813ce3fa-2c29-4b24-bd94-fe7194d1ac38 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:10:53.684868 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-813ce3fa-2c29-4b24-bd94-fe7194d1ac38 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:10:53.696392 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813ce3fa-2c29-4b24-bd94-fe7194d1ac38 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:10:53.722024 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:10:53.793314 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4197c873-ff85-4f3e-82b0-00966471b9b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:10:53.793994 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4197c873-ff85-4f3e-82b0-00966471b9b1 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:10:53.794282 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4197c873-ff85-4f3e-82b0-00966471b9b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:10:53.794565 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4197c873-ff85-4f3e-82b0-00966471b9b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:10:53.795294 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4197c873-ff85-4f3e-82b0-00966471b9b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:10:53.796353 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: WARNING py.warnings [None req-4197c873-ff85-4f3e-82b0-00966471b9b1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 25 12:10:53.796353 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: warnings.warn(msg) Jul 25 12:10:53.796353 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: Jul 25 12:10:53.797242 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4197c873-ff85-4f3e-82b0-00966471b9b1 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:10:53.809437 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 754/1500] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Jul 25 12:10:53 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:10:53.813742 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2473382-c956-4366-9407-4f4a15e27368 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:10:53.815273 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-b2473382-c956-4366-9407-4f4a15e27368 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:10:53.849060 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2473382-c956-4366-9407-4f4a15e27368 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:10:53.940784 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459d6b79-599d-4b49-b3cb-35246727e7b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:10:53.941664 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459d6b79-599d-4b49-b3cb-35246727e7b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:10:53.941947 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459d6b79-599d-4b49-b3cb-35246727e7b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:10:53.942222 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459d6b79-599d-4b49-b3cb-35246727e7b3 admin admin] PATH_INFO: `/v3/projects/95cc02a30ed54e10abb6f02cf3e15610/users/55285fcd46ba42809fc7f3b068db0f81/roles/472313ae69174578824df0bed6390744` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:10:53.965128 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459d6b79-599d-4b49-b3cb-35246727e7b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=95cc02a30ed54e10abb6f02cf3e15610, user_id=55285fcd46ba42809fc7f3b068db0f81, role_id=472313ae69174578824df0bed6390744)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:10:53.966160 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: WARNING py.warnings [None req-459d6b79-599d-4b49-b3cb-35246727e7b3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 25 12:10:53.966160 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: warnings.warn(msg) Jul 25 12:10:53.966160 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: Jul 25 12:10:53.968125 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459d6b79-599d-4b49-b3cb-35246727e7b3 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:10:53.992308 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 747/1501] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Jul 25 12:10:53 2022] PUT /identity/v3/projects/95cc02a30ed54e10abb6f02cf3e15610/users/55285fcd46ba42809fc7f3b068db0f81/roles/472313ae69174578824df0bed6390744 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:10:54.004860 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4108b90-f849-43b2-aeaa-aa14b019ec75 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:10:54.005158 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4108b90-f849-43b2-aeaa-aa14b019ec75 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:10:54.005444 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4108b90-f849-43b2-aeaa-aa14b019ec75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:10:54.062521 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-e4108b90-f849-43b2-aeaa-aa14b019ec75 None None] MFA Rules not processed for user `55285fcd46ba42809fc7f3b068db0f81`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:10:54.063774 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-e4108b90-f849-43b2-aeaa-aa14b019ec75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:10:54.140806 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 755/1502] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:10:54 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:10:54.154530 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135e5f8a-4f9e-4581-8d6b-fec563447bff None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:10:54.154825 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135e5f8a-4f9e-4581-8d6b-fec563447bff None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:10:54.155112 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135e5f8a-4f9e-4581-8d6b-fec563447bff None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:10:54.156514 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 748/1503] 10.222.0.42 () {54 vars in 964 bytes} [Mon Jul 25 12:10:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:10:54.161622 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6959e49-40ae-4de0-8093-a853b1af3562 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:10:54.161920 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6959e49-40ae-4de0-8093-a853b1af3562 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:10:54.162207 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6959e49-40ae-4de0-8093-a853b1af3562 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:10:54.194450 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:10:54.239351 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-f6959e49-40ae-4de0-8093-a853b1af3562 None None] MFA Rules not processed for user `6f89ce6f0cf04373a414d927fa0f8a7f`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:10:54.240610 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-f6959e49-40ae-4de0-8093-a853b1af3562 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:10:54.317013 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 756/1504] 10.222.0.42 () {58 vars in 1061 bytes} [Mon Jul 25 12:10:54 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:10:54.325424 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135e5f8a-4f9e-4581-8d6b-fec563447bff None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:10:54.326961 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-135e5f8a-4f9e-4581-8d6b-fec563447bff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:10:54.380458 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:10:54.435701 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135e5f8a-4f9e-4581-8d6b-fec563447bff None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:10:54.506839 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a262ad-fdbd-473a-af3c-e05cb20c1173 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f89ce6f0cf04373a414d927fa0f8a7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:10:54.507481 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a262ad-fdbd-473a-af3c-e05cb20c1173 service cinder] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:10:54.507798 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a262ad-fdbd-473a-af3c-e05cb20c1173 service cinder] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:10:54.508082 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a262ad-fdbd-473a-af3c-e05cb20c1173 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:10:54.509718 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-39a262ad-fdbd-473a-af3c-e05cb20c1173 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:10:54.614990 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a262ad-fdbd-473a-af3c-e05cb20c1173 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:10:54.616454 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a262ad-fdbd-473a-af3c-e05cb20c1173 service cinder] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:10:54.620767 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:10:54.637884 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 749/1505] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:10:54 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:10:54.683611 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26070c63-576a-4053-bb19-0c756adaa7ac None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:10:54.684052 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26070c63-576a-4053-bb19-0c756adaa7ac None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:10:54.684188 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26070c63-576a-4053-bb19-0c756adaa7ac None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:10:54.685471 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 757/1506] 10.222.0.42 () {52 vars in 830 bytes} [Mon Jul 25 12:10:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:10:54.689075 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a262ad-fdbd-473a-af3c-e05cb20c1173 service cinder] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:10:54.690302 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-39a262ad-fdbd-473a-af3c-e05cb20c1173 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:10:54.789450 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a262ad-fdbd-473a-af3c-e05cb20c1173 service cinder] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:10:54.882099 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bd377b-15e4-43d8-9427-f55dacf53566 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:10:54.882895 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bd377b-15e4-43d8-9427-f55dacf53566 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:10:54.883179 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bd377b-15e4-43d8-9427-f55dacf53566 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:10:54.883458 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bd377b-15e4-43d8-9427-f55dacf53566 admin admin] PATH_INFO: `/v3/users/55285fcd46ba42809fc7f3b068db0f81` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:10:54.895572 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24bd377b-15e4-43d8-9427-f55dacf53566 admin admin] RBAC: Authorizing `identity:delete_user(user_id=55285fcd46ba42809fc7f3b068db0f81)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:10:54.896602 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: WARNING py.warnings [None req-24bd377b-15e4-43d8-9427-f55dacf53566 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 25 12:10:54.896602 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: warnings.warn(msg) Jul 25 12:10:54.896602 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: Jul 25 12:10:54.897807 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24bd377b-15e4-43d8-9427-f55dacf53566 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:10:54.908021 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:10:54.968291 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-24bd377b-15e4-43d8-9427-f55dacf53566 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '55285fcd46ba42809fc7f3b068db0f81'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:10:54.980933 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-24bd377b-15e4-43d8-9427-f55dacf53566 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '55285fcd46ba42809fc7f3b068db0f81'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:10:54.990426 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-24bd377b-15e4-43d8-9427-f55dacf53566 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '55285fcd46ba42809fc7f3b068db0f81'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:10:55.008559 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-24bd377b-15e4-43d8-9427-f55dacf53566 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '55285fcd46ba42809fc7f3b068db0f81'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:10:55.009090 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-24bd377b-15e4-43d8-9427-f55dacf53566 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '55285fcd46ba42809fc7f3b068db0f81'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:10:55.014820 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 750/1507] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Jul 25 12:10:54 2022] DELETE /identity/v3/users/55285fcd46ba42809fc7f3b068db0f81 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:10:55.019345 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7da59d-2e37-454b-a1ae-19c189fc66ca None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:10:55.019675 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7da59d-2e37-454b-a1ae-19c189fc66ca None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:10:55.019959 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7da59d-2e37-454b-a1ae-19c189fc66ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:10:55.078134 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-6a7da59d-2e37-454b-a1ae-19c189fc66ca None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:10:55.079356 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-6a7da59d-2e37-454b-a1ae-19c189fc66ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:10:55.154005 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:10:55.167797 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 758/1508] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:10:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 12:10:55.177332 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bd377b-15e4-43d8-9427-f55dacf53566 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:10:55.178565 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-24bd377b-15e4-43d8-9427-f55dacf53566 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:10:55.287416 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bd377b-15e4-43d8-9427-f55dacf53566 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:10:55.341988 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:10:55.363058 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38b1107-3913-4ad0-99c2-b450023ad323 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51733781201243e8abf9eca39b207750', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:10:55.363757 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38b1107-3913-4ad0-99c2-b450023ad323 service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:10:55.364047 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38b1107-3913-4ad0-99c2-b450023ad323 service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:10:55.364331 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38b1107-3913-4ad0-99c2-b450023ad323 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:10:55.365982 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-c38b1107-3913-4ad0-99c2-b450023ad323 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:10:55.472609 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38b1107-3913-4ad0-99c2-b450023ad323 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:10:55.473945 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38b1107-3913-4ad0-99c2-b450023ad323 service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:10:55.491724 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 751/1509] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:10:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:10:55.849694 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7da59d-2e37-454b-a1ae-19c189fc66ca None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:10:55.851307 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-6a7da59d-2e37-454b-a1ae-19c189fc66ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:10:55.863910 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7da59d-2e37-454b-a1ae-19c189fc66ca None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:10:55.960855 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69ce79d-aeb4-42da-ab80-52b0a6d36c3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:10:55.961718 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69ce79d-aeb4-42da-ab80-52b0a6d36c3d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:10:55.962003 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69ce79d-aeb4-42da-ab80-52b0a6d36c3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:10:55.962281 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69ce79d-aeb4-42da-ab80-52b0a6d36c3d admin admin] PATH_INFO: `/v3/projects/95cc02a30ed54e10abb6f02cf3e15610` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:10:55.971939 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69ce79d-aeb4-42da-ab80-52b0a6d36c3d admin admin] RBAC: Authorizing `identity:delete_project(project_id=95cc02a30ed54e10abb6f02cf3e15610)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:10:55.972959 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: WARNING py.warnings [None req-c69ce79d-aeb4-42da-ab80-52b0a6d36c3d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 25 12:10:55.972959 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: warnings.warn(msg) Jul 25 12:10:55.972959 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: Jul 25 12:10:55.974146 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69ce79d-aeb4-42da-ab80-52b0a6d36c3d admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:10:56.030314 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-c69ce79d-aeb4-42da-ab80-52b0a6d36c3d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '95cc02a30ed54e10abb6f02cf3e15610', 'request_id': 'req-c69ce79d-aeb4-42da-ab80-52b0a6d36c3d'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:10:56.039134 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-c69ce79d-aeb4-42da-ab80-52b0a6d36c3d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '95cc02a30ed54e10abb6f02cf3e15610', 'request_id': 'req-c69ce79d-aeb4-42da-ab80-52b0a6d36c3d'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:10:56.045897 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-c69ce79d-aeb4-42da-ab80-52b0a6d36c3d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '95cc02a30ed54e10abb6f02cf3e15610', 'request_id': 'req-c69ce79d-aeb4-42da-ab80-52b0a6d36c3d'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:10:56.076579 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-c69ce79d-aeb4-42da-ab80-52b0a6d36c3d admin admin] The token cache is being invalidate because project 95cc02a30ed54e10abb6f02cf3e15610 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80298) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 12:10:56.076866 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-c69ce79d-aeb4-42da-ab80-52b0a6d36c3d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:10:56.077386 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-c69ce79d-aeb4-42da-ab80-52b0a6d36c3d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:10:56.078710 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 759/1510] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Jul 25 12:10:55 2022] DELETE /identity/v3/projects/95cc02a30ed54e10abb6f02cf3e15610 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:11:05.489458 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=62 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:11:06.074613 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:12:36.963308 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f564942-2c49-47a9-91ba-800fddf0ce24 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:12:36.963936 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f564942-2c49-47a9-91ba-800fddf0ce24 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:12:36.963936 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f564942-2c49-47a9-91ba-800fddf0ce24 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:12:36.965283 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 752/1511] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 25 12:12:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:12:36.970356 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783e25c0-187d-431a-b837-00e648395afb None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:12:36.970806 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783e25c0-187d-431a-b837-00e648395afb None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:12:36.970951 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783e25c0-187d-431a-b837-00e648395afb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:12:37.051636 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-783e25c0-187d-431a-b837-00e648395afb None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:12:37.052871 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-783e25c0-187d-431a-b837-00e648395afb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:12:37.156637 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 760/1512] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 25 12:12:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:12:37.312550 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f564942-2c49-47a9-91ba-800fddf0ce24 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:12:37.314635 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-5f564942-2c49-47a9-91ba-800fddf0ce24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:12:37.429779 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f564942-2c49-47a9-91ba-800fddf0ce24 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:12:37.436119 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1ccd29-c05e-4862-8bb6-5b43f26488b1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f67142a1193d47c7bd33d7040268233c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:12:37.436760 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1ccd29-c05e-4862-8bb6-5b43f26488b1 service nova] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:12:37.437047 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1ccd29-c05e-4862-8bb6-5b43f26488b1 service nova] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:12:37.437327 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1ccd29-c05e-4862-8bb6-5b43f26488b1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:12:37.445407 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1ccd29-c05e-4862-8bb6-5b43f26488b1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:12:37.447011 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1ccd29-c05e-4862-8bb6-5b43f26488b1 service nova] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:12:37.456281 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 753/1513] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:12:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:12:38.287335 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6caeb8-4af3-4e0f-bb4e-df7ef57360ba None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:12:38.287780 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6caeb8-4af3-4e0f-bb4e-df7ef57360ba None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:12:38.287937 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6caeb8-4af3-4e0f-bb4e-df7ef57360ba None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:12:38.289319 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 761/1514] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 25 12:12:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:12:38.294108 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b16ccc9-4197-47cc-8d22-554a3537f1d9 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:12:38.294521 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b16ccc9-4197-47cc-8d22-554a3537f1d9 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:12:38.294805 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b16ccc9-4197-47cc-8d22-554a3537f1d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:12:38.314633 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-4b16ccc9-4197-47cc-8d22-554a3537f1d9 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:12:38.315880 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-4b16ccc9-4197-47cc-8d22-554a3537f1d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:12:38.324732 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 754/1515] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 25 12:12:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:12:38.476098 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6caeb8-4af3-4e0f-bb4e-df7ef57360ba None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:12:38.490476 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6caeb8-4af3-4e0f-bb4e-df7ef57360ba None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:12:38.496494 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e0eeab-fe02-4a76-bae1-081684a1dfc9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f67142a1193d47c7bd33d7040268233c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:12:38.497140 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e0eeab-fe02-4a76-bae1-081684a1dfc9 service nova] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:12:38.497426 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e0eeab-fe02-4a76-bae1-081684a1dfc9 service nova] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:12:38.497709 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e0eeab-fe02-4a76-bae1-081684a1dfc9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:12:38.505809 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e0eeab-fe02-4a76-bae1-081684a1dfc9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:12:38.507398 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e0eeab-fe02-4a76-bae1-081684a1dfc9 service nova] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:12:38.517412 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 762/1516] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:12:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:12:39.300601 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67686561-7601-42fd-96fc-acc7a5972865 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:12:39.301167 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67686561-7601-42fd-96fc-acc7a5972865 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:12:39.301167 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67686561-7601-42fd-96fc-acc7a5972865 None None] PATH_INFO: `/v3` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:12:39.302465 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 755/1517] 10.222.0.39 () {54 vars in 925 bytes} [Mon Jul 25 12:12:39 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:12:39.308459 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dd682b-25f5-4b53-b23d-daed13ed3521 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:12:39.308901 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dd682b-25f5-4b53-b23d-daed13ed3521 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:12:39.309007 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dd682b-25f5-4b53-b23d-daed13ed3521 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:12:39.367620 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-a6dd682b-25f5-4b53-b23d-daed13ed3521 None None] MFA Rules not processed for user `8012afab6cac4358ab878732e58d0689`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:12:39.368841 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-a6dd682b-25f5-4b53-b23d-daed13ed3521 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:12:39.385778 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:12:39.439864 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 763/1518] 10.222.0.39 () {58 vars in 1015 bytes} [Mon Jul 25 12:12:39 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:12:43.285300 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6ed2bb-d503-4221-8572-28c6fbe5f626 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:12:43.285871 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6ed2bb-d503-4221-8572-28c6fbe5f626 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:12:43.285871 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6ed2bb-d503-4221-8572-28c6fbe5f626 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:12:43.287464 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 756/1519] 10.222.0.42 () {54 vars in 961 bytes} [Mon Jul 25 12:12:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:12:43.293388 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06773a8e-f119-4898-bd34-73ab8ebf6257 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:12:43.293827 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06773a8e-f119-4898-bd34-73ab8ebf6257 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:12:43.293967 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06773a8e-f119-4898-bd34-73ab8ebf6257 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:12:43.313896 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-06773a8e-f119-4898-bd34-73ab8ebf6257 None None] MFA Rules not processed for user `8012afab6cac4358ab878732e58d0689`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:12:43.315110 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-06773a8e-f119-4898-bd34-73ab8ebf6257 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:12:43.324059 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 764/1520] 10.222.0.42 () {58 vars in 1058 bytes} [Mon Jul 25 12:12:43 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:12:43.331424 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6ed2bb-d503-4221-8572-28c6fbe5f626 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:12:43.344603 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6ed2bb-d503-4221-8572-28c6fbe5f626 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:12:43.349402 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf383e5-6aca-4ff2-9788-e9d6074a4a2f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8012afab6cac4358ab878732e58d0689', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:12:43.350032 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf383e5-6aca-4ff2-9788-e9d6074a4a2f service placement] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:12:43.350315 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf383e5-6aca-4ff2-9788-e9d6074a4a2f service placement] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:12:43.350594 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf383e5-6aca-4ff2-9788-e9d6074a4a2f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:12:43.359300 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf383e5-6aca-4ff2-9788-e9d6074a4a2f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:12:43.360941 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf383e5-6aca-4ff2-9788-e9d6074a4a2f service placement] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:12:43.371282 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 757/1521] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:12:43 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 12:12:49.738909 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e117a650-db71-4395-98be-572a6bd2b37a None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:12:49.739524 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e117a650-db71-4395-98be-572a6bd2b37a None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:12:49.739524 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e117a650-db71-4395-98be-572a6bd2b37a None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:12:49.740926 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 765/1522] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 25 12:12:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:12:49.745683 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e29b19-19bb-4974-bc59-1c97db21173f None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:12:49.746070 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e29b19-19bb-4974-bc59-1c97db21173f None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:12:49.746261 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e29b19-19bb-4974-bc59-1c97db21173f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:12:49.765317 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-11e29b19-19bb-4974-bc59-1c97db21173f None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:12:49.766519 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-11e29b19-19bb-4974-bc59-1c97db21173f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:12:49.774986 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 758/1523] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 25 12:12:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:12:49.931215 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e117a650-db71-4395-98be-572a6bd2b37a None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:12:49.946792 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e117a650-db71-4395-98be-572a6bd2b37a None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:12:49.953188 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6055e5c-4600-4eb1-b58d-6af458bc4128 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f67142a1193d47c7bd33d7040268233c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:12:49.953844 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6055e5c-4600-4eb1-b58d-6af458bc4128 service nova] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:12:49.954136 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6055e5c-4600-4eb1-b58d-6af458bc4128 service nova] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:12:49.954602 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6055e5c-4600-4eb1-b58d-6af458bc4128 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:12:49.966115 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6055e5c-4600-4eb1-b58d-6af458bc4128 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:12:49.969085 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6055e5c-4600-4eb1-b58d-6af458bc4128 service nova] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:12:49.979194 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 766/1524] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:12:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:12:50.800330 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71cc012-10ab-4776-833a-b4ce25856166 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:12:50.800810 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71cc012-10ab-4776-833a-b4ce25856166 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:12:50.800926 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71cc012-10ab-4776-833a-b4ce25856166 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:12:50.802259 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 759/1525] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 25 12:12:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:12:50.807033 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77b86a6-1dc3-4d5c-acf8-05f2564a821e None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:12:50.807450 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77b86a6-1dc3-4d5c-acf8-05f2564a821e None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:12:50.807626 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77b86a6-1dc3-4d5c-acf8-05f2564a821e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:12:50.829116 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-d77b86a6-1dc3-4d5c-acf8-05f2564a821e None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:12:50.830355 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-d77b86a6-1dc3-4d5c-acf8-05f2564a821e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:12:50.839214 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 767/1526] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 25 12:12:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:12:50.989805 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71cc012-10ab-4776-833a-b4ce25856166 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:12:51.004685 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71cc012-10ab-4776-833a-b4ce25856166 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:12:51.010733 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def3f449-ddcf-414d-991f-44c413be464d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f67142a1193d47c7bd33d7040268233c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:12:51.011367 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def3f449-ddcf-414d-991f-44c413be464d service nova] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:12:51.011691 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def3f449-ddcf-414d-991f-44c413be464d service nova] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:12:51.011974 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def3f449-ddcf-414d-991f-44c413be464d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:12:51.019984 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def3f449-ddcf-414d-991f-44c413be464d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:12:51.021598 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def3f449-ddcf-414d-991f-44c413be464d service nova] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:12:51.030938 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 760/1527] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:12:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:12:54.967689 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fee953-3dfe-4a4f-ade7-535afdd476e6 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:12:54.968270 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fee953-3dfe-4a4f-ade7-535afdd476e6 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:12:54.968270 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fee953-3dfe-4a4f-ade7-535afdd476e6 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:12:54.969472 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 768/1528] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 25 12:12:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:12:54.974626 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d7c498-b2ba-45b3-bc83-b3d44a49e23a None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:12:54.975083 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d7c498-b2ba-45b3-bc83-b3d44a49e23a None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:12:54.975190 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d7c498-b2ba-45b3-bc83-b3d44a49e23a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:12:54.995115 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-e7d7c498-b2ba-45b3-bc83-b3d44a49e23a None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:12:54.996353 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-e7d7c498-b2ba-45b3-bc83-b3d44a49e23a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:12:55.004180 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 761/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 25 12:12:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 12:12:55.078293 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fee953-3dfe-4a4f-ade7-535afdd476e6 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:12:55.080008 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-61fee953-3dfe-4a4f-ade7-535afdd476e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:12:55.158518 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fee953-3dfe-4a4f-ade7-535afdd476e6 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:12:55.162367 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49891748-e2f9-40a2-9ace-d79bd23c4337 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51733781201243e8abf9eca39b207750', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:12:55.163007 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49891748-e2f9-40a2-9ace-d79bd23c4337 service neutron] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:12:55.163296 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49891748-e2f9-40a2-9ace-d79bd23c4337 service neutron] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:12:55.163603 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49891748-e2f9-40a2-9ace-d79bd23c4337 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:12:55.172479 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49891748-e2f9-40a2-9ace-d79bd23c4337 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:12:55.173897 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49891748-e2f9-40a2-9ace-d79bd23c4337 service neutron] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:12:55.183262 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 769/1530] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:12:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:12:57.179395 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1702f7b6-fe18-4c14-9ca2-1e56ae2da729 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:12:57.179989 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1702f7b6-fe18-4c14-9ca2-1e56ae2da729 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:12:57.179989 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1702f7b6-fe18-4c14-9ca2-1e56ae2da729 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:12:57.181440 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 762/1531] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 25 12:12:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:12:57.187230 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1c18dc-d282-4abe-9564-50f6af476467 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:12:57.187678 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1c18dc-d282-4abe-9564-50f6af476467 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:12:57.187825 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1c18dc-d282-4abe-9564-50f6af476467 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:12:57.207404 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-6a1c18dc-d282-4abe-9564-50f6af476467 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:12:57.208656 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-6a1c18dc-d282-4abe-9564-50f6af476467 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:12:57.216561 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 770/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 25 12:12:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:12:57.251786 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1702f7b6-fe18-4c14-9ca2-1e56ae2da729 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:12:57.264733 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1702f7b6-fe18-4c14-9ca2-1e56ae2da729 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:12:57.268928 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525a0aaf-dd82-4801-af76-af58f99a69f1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51733781201243e8abf9eca39b207750', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:12:57.269572 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525a0aaf-dd82-4801-af76-af58f99a69f1 service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:12:57.269861 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525a0aaf-dd82-4801-af76-af58f99a69f1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:12:57.270145 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525a0aaf-dd82-4801-af76-af58f99a69f1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:12:57.279443 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525a0aaf-dd82-4801-af76-af58f99a69f1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:12:57.280912 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525a0aaf-dd82-4801-af76-af58f99a69f1 service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:12:57.290903 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 763/1533] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:12:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:13:00.647229 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc8c015-3f96-4334-bb18-6d871aaf01df None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:13:00.647791 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc8c015-3f96-4334-bb18-6d871aaf01df None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:13:00.647791 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc8c015-3f96-4334-bb18-6d871aaf01df None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:13:00.649120 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 771/1534] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 25 12:13:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 12:13:00.654842 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb09afe-675a-4f7b-9c46-68ade574fa84 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:13:00.655266 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb09afe-675a-4f7b-9c46-68ade574fa84 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:13:00.655415 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb09afe-675a-4f7b-9c46-68ade574fa84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:13:00.675570 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-beb09afe-675a-4f7b-9c46-68ade574fa84 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:13:00.676777 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-beb09afe-675a-4f7b-9c46-68ade574fa84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:13:00.684581 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 764/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 25 12:13:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:13:00.990040 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc8c015-3f96-4334-bb18-6d871aaf01df None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:13:00.991749 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-8cc8c015-3f96-4334-bb18-6d871aaf01df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:13:01.055454 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:13:01.068847 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc8c015-3f96-4334-bb18-6d871aaf01df None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:13:01.072980 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522ab778-ae49-49b7-902c-77e09969468a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f89ce6f0cf04373a414d927fa0f8a7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:13:01.073622 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522ab778-ae49-49b7-902c-77e09969468a service cinder] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:13:01.073912 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522ab778-ae49-49b7-902c-77e09969468a service cinder] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:13:01.074196 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522ab778-ae49-49b7-902c-77e09969468a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:13:01.083100 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522ab778-ae49-49b7-902c-77e09969468a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:13:01.084540 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522ab778-ae49-49b7-902c-77e09969468a service cinder] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:13:01.094653 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 772/1536] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:13:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:13:03.097516 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feeedbdd-cda1-490b-b1f1-f73d16a67f05 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:13:03.098074 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feeedbdd-cda1-490b-b1f1-f73d16a67f05 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:13:03.098074 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feeedbdd-cda1-490b-b1f1-f73d16a67f05 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:13:03.099497 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 765/1537] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 25 12:13:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:13:03.104835 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff964ed3-188d-4657-84ce-017db999bac5 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:13:03.105238 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff964ed3-188d-4657-84ce-017db999bac5 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:13:03.105441 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff964ed3-188d-4657-84ce-017db999bac5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:13:03.127407 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-ff964ed3-188d-4657-84ce-017db999bac5 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:13:03.129398 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-ff964ed3-188d-4657-84ce-017db999bac5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:13:03.139445 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 773/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 25 12:13:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:13:03.441143 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feeedbdd-cda1-490b-b1f1-f73d16a67f05 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:13:03.442914 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-feeedbdd-cda1-490b-b1f1-f73d16a67f05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:13:03.455794 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feeedbdd-cda1-490b-b1f1-f73d16a67f05 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:13:03.459358 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d0a4d1-15f9-42c9-9509-4d9f8c8cf5ef service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f89ce6f0cf04373a414d927fa0f8a7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:13:03.460034 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d0a4d1-15f9-42c9-9509-4d9f8c8cf5ef service cinder] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:13:03.460327 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d0a4d1-15f9-42c9-9509-4d9f8c8cf5ef service cinder] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:13:03.460616 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d0a4d1-15f9-42c9-9509-4d9f8c8cf5ef service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:13:03.470382 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d0a4d1-15f9-42c9-9509-4d9f8c8cf5ef service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:13:03.471844 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d0a4d1-15f9-42c9-9509-4d9f8c8cf5ef service cinder] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:13:03.481764 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 766/1539] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:13:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:13:13.120668 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=11 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:13:13.477115 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=89 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:14:33.270580 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75450288-415a-4ad0-8a16-e45c31be4511 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:14:33.271829 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75450288-415a-4ad0-8a16-e45c31be4511 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:14:33.272313 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75450288-415a-4ad0-8a16-e45c31be4511 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:14:33.303997 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-75450288-415a-4ad0-8a16-e45c31be4511 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:14:33.305252 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-75450288-415a-4ad0-8a16-e45c31be4511 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:14:33.316676 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 774/1540] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:14:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:14:33.326695 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9a5d5e-cfc2-4c8c-bf04-6501714c28a8 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:14:33.327277 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9a5d5e-cfc2-4c8c-bf04-6501714c28a8 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:14:33.327277 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9a5d5e-cfc2-4c8c-bf04-6501714c28a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:14:33.352194 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-cf9a5d5e-cfc2-4c8c-bf04-6501714c28a8 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:14:33.353939 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-cf9a5d5e-cfc2-4c8c-bf04-6501714c28a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:14:33.362042 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f95401-da1b-4d3c-bfab-93b13264097e None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:14:33.362355 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f95401-da1b-4d3c-bfab-93b13264097e None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:14:33.362614 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f95401-da1b-4d3c-bfab-93b13264097e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:14:33.372205 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 767/1541] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:14:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:14:33.377156 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9a5d5e-cfc2-4c8c-bf04-6501714c28a8 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:14:33.387927 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9a5d5e-cfc2-4c8c-bf04-6501714c28a8 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:14:33.395037 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-b8f95401-da1b-4d3c-bfab-93b13264097e None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:14:33.396600 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-b8f95401-da1b-4d3c-bfab-93b13264097e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:14:33.404449 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8e50b6-3d93-4bbd-9713-17e00271c31d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:14:33.405525 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8e50b6-3d93-4bbd-9713-17e00271c31d admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:14:33.406193 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8e50b6-3d93-4bbd-9713-17e00271c31d admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:14:33.406696 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8e50b6-3d93-4bbd-9713-17e00271c31d admin admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:14:33.407725 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8e50b6-3d93-4bbd-9713-17e00271c31d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:14:33.408263 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8e50b6-3d93-4bbd-9713-17e00271c31d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:14:33.409672 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: WARNING py.warnings [None req-5a8e50b6-3d93-4bbd-9713-17e00271c31d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 25 12:14:33.409672 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: warnings.warn(msg) Jul 25 12:14:33.409672 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: Jul 25 12:14:33.411256 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8e50b6-3d93-4bbd-9713-17e00271c31d admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:14:33.416096 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 775/1542] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:14:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:14:33.429209 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6bf6a1-6a3f-4076-9745-2ffafba240fa None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:14:33.429565 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6bf6a1-6a3f-4076-9745-2ffafba240fa None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:14:33.429782 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6bf6a1-6a3f-4076-9745-2ffafba240fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:14:33.435435 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 768/1543] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Jul 25 12:14:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 12:14:33.444090 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8e50b6-3d93-4bbd-9713-17e00271c31d admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:14:33.464574 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8e50b6-3d93-4bbd-9713-17e00271c31d admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:14:33.466124 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-db6bf6a1-6a3f-4076-9745-2ffafba240fa None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:14:33.467362 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-db6bf6a1-6a3f-4076-9745-2ffafba240fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:14:33.476149 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653303e1-1ed1-4700-99ea-3e5b5f95908e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:14:33.476817 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653303e1-1ed1-4700-99ea-3e5b5f95908e admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:14:33.477192 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653303e1-1ed1-4700-99ea-3e5b5f95908e admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:14:33.477479 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653303e1-1ed1-4700-99ea-3e5b5f95908e admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:14:33.478189 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653303e1-1ed1-4700-99ea-3e5b5f95908e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:14:33.483754 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653303e1-1ed1-4700-99ea-3e5b5f95908e admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:14:33.487862 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 776/1544] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:14:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:14:33.496762 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6bf6a1-6a3f-4076-9745-2ffafba240fa None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:14:33.519795 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6bf6a1-6a3f-4076-9745-2ffafba240fa None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:14:33.533418 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8240a9-d895-413a-83f1-a3402a1fa133 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:14:33.534033 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8240a9-d895-413a-83f1-a3402a1fa133 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:14:33.534354 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8240a9-d895-413a-83f1-a3402a1fa133 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:14:33.534598 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8240a9-d895-413a-83f1-a3402a1fa133 admin admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:14:33.535363 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8240a9-d895-413a-83f1-a3402a1fa133 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:14:33.537343 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8240a9-d895-413a-83f1-a3402a1fa133 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:14:33.539123 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8240a9-d895-413a-83f1-a3402a1fa133 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:14:33.546065 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 769/1545] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Jul 25 12:14:33 2022] POST /identity/v3/projects => generated 383 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 12:14:33.550667 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653303e1-1ed1-4700-99ea-3e5b5f95908e admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:14:33.561196 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 777/1546] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Jul 25 12:14:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:14:33.569323 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8240a9-d895-413a-83f1-a3402a1fa133 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:14:33.589006 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8240a9-d895-413a-83f1-a3402a1fa133 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:14:33.633131 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653303e1-1ed1-4700-99ea-3e5b5f95908e admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:14:33.643847 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886ec46b-c219-418e-aafd-8b4e89ccf537 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:14:33.644532 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886ec46b-c219-418e-aafd-8b4e89ccf537 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:14:33.644819 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886ec46b-c219-418e-aafd-8b4e89ccf537 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:14:33.645099 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886ec46b-c219-418e-aafd-8b4e89ccf537 admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:14:33.645802 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886ec46b-c219-418e-aafd-8b4e89ccf537 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:14:33.646836 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: WARNING py.warnings [None req-886ec46b-c219-418e-aafd-8b4e89ccf537 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 25 12:14:33.646836 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: warnings.warn(msg) Jul 25 12:14:33.646836 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: Jul 25 12:14:33.648075 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886ec46b-c219-418e-aafd-8b4e89ccf537 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:14:33.687938 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 770/1547] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Jul 25 12:14:33 2022] POST /identity/v3/users => generated 292 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:14:33.688888 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886ec46b-c219-418e-aafd-8b4e89ccf537 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:14:33.699936 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886ec46b-c219-418e-aafd-8b4e89ccf537 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:14:33.714969 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63336acd-e117-47fb-99ce-6995fe81e36e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:14:33.715679 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63336acd-e117-47fb-99ce-6995fe81e36e admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:14:33.715974 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63336acd-e117-47fb-99ce-6995fe81e36e admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:14:33.716259 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63336acd-e117-47fb-99ce-6995fe81e36e admin admin] PATH_INFO: `/v3/projects` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:14:33.716981 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63336acd-e117-47fb-99ce-6995fe81e36e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:14:33.718018 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: WARNING py.warnings [None req-63336acd-e117-47fb-99ce-6995fe81e36e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 25 12:14:33.718018 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: warnings.warn(msg) Jul 25 12:14:33.718018 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: Jul 25 12:14:33.719227 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63336acd-e117-47fb-99ce-6995fe81e36e admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:14:33.721813 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f591f1-f9a5-438b-a7b0-6ad4fd9e6492 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:14:33.722501 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f591f1-f9a5-438b-a7b0-6ad4fd9e6492 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:14:33.723015 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f591f1-f9a5-438b-a7b0-6ad4fd9e6492 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:14:33.723434 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f591f1-f9a5-438b-a7b0-6ad4fd9e6492 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:14:33.724213 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f591f1-f9a5-438b-a7b0-6ad4fd9e6492 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:14:33.725825 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f591f1-f9a5-438b-a7b0-6ad4fd9e6492 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:14:33.749622 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 771/1548] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Jul 25 12:14:33 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:14:33.752128 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886dacad-2889-4c29-810d-fa9fb45566ae None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:14:33.752431 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886dacad-2889-4c29-810d-fa9fb45566ae None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:14:33.752713 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886dacad-2889-4c29-810d-fa9fb45566ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:14:33.797509 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 778/1549] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Jul 25 12:14:33 2022] POST /identity/v3/projects => generated 389 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 12:14:33.798721 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13cfec4-e6f5-47ff-a2f6-ec7ce29df9cb None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:14:33.799028 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13cfec4-e6f5-47ff-a2f6-ec7ce29df9cb None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:14:33.799309 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13cfec4-e6f5-47ff-a2f6-ec7ce29df9cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:14:33.808213 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-886dacad-2889-4c29-810d-fa9fb45566ae None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:14:33.810577 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-886dacad-2889-4c29-810d-fa9fb45566ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:14:33.834658 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-d13cfec4-e6f5-47ff-a2f6-ec7ce29df9cb None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:14:33.835953 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-d13cfec4-e6f5-47ff-a2f6-ec7ce29df9cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:14:33.887845 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 772/1550] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:14:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:14:33.888978 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886dacad-2889-4c29-810d-fa9fb45566ae None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:14:33.898973 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886dacad-2889-4c29-810d-fa9fb45566ae None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:14:33.901964 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 779/1551] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:14:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:14:33.903918 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e14ae02-ae12-4147-9ce0-1361c41b41c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:14:33.904586 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e14ae02-ae12-4147-9ce0-1361c41b41c3 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:14:33.904839 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13cfec4-e6f5-47ff-a2f6-ec7ce29df9cb None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:14:33.904982 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e14ae02-ae12-4147-9ce0-1361c41b41c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:14:33.905184 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e14ae02-ae12-4147-9ce0-1361c41b41c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:14:33.906026 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e14ae02-ae12-4147-9ce0-1361c41b41c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:14:33.907931 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e14ae02-ae12-4147-9ce0-1361c41b41c3 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:14:33.914383 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13cfec4-e6f5-47ff-a2f6-ec7ce29df9cb None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:14:33.918676 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 773/1552] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Jul 25 12:14:33 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:14:33.918999 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e2c3b8-0157-47a9-ab9e-6de0cb1f3e93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:14:33.922647 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e2c3b8-0157-47a9-ab9e-6de0cb1f3e93 admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:14:33.922647 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e2c3b8-0157-47a9-ab9e-6de0cb1f3e93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:14:33.922894 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e8fb7e-c4b9-4d6c-8d50-e8c17aa6f31e None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:14:33.922987 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e2c3b8-0157-47a9-ab9e-6de0cb1f3e93 admin admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:14:33.923083 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e8fb7e-c4b9-4d6c-8d50-e8c17aa6f31e None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:14:33.923083 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e8fb7e-c4b9-4d6c-8d50-e8c17aa6f31e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:14:33.925817 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e2c3b8-0157-47a9-ab9e-6de0cb1f3e93 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:14:33.925817 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e2c3b8-0157-47a9-ab9e-6de0cb1f3e93 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:14:33.941674 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-78e8fb7e-c4b9-4d6c-8d50-e8c17aa6f31e None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:14:33.942916 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-78e8fb7e-c4b9-4d6c-8d50-e8c17aa6f31e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:14:33.949473 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 780/1553] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Jul 25 12:14:33 2022] POST /identity/v3/users => generated 295 bytes in 42 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:14:33.949473 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edcb2aa-10a7-4802-a89d-efd287c76673 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:14:33.949473 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edcb2aa-10a7-4802-a89d-efd287c76673 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:14:33.949473 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edcb2aa-10a7-4802-a89d-efd287c76673 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:14:33.954073 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 774/1554] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:14:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:14:33.954073 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfe933c-af54-4f56-b3a5-442b7d956f64 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:14:33.954073 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfe933c-af54-4f56-b3a5-442b7d956f64 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:14:33.954073 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfe933c-af54-4f56-b3a5-442b7d956f64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:14:33.972098 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-9edcb2aa-10a7-4802-a89d-efd287c76673 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:14:33.973371 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-9edcb2aa-10a7-4802-a89d-efd287c76673 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:14:33.974957 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-bbfe933c-af54-4f56-b3a5-442b7d956f64 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:14:33.976219 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-bbfe933c-af54-4f56-b3a5-442b7d956f64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:14:33.982032 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 781/1555] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:14:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:14:33.983078 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b5513a-5660-486a-97ed-e4860321bc49 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:14:33.983370 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b5513a-5660-486a-97ed-e4860321bc49 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:14:33.983687 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b5513a-5660-486a-97ed-e4860321bc49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:14:33.984920 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 775/1556] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:14:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:14:33.986247 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03495ca-652c-45f6-a820-06fd6d1d4362 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:14:33.986559 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03495ca-652c-45f6-a820-06fd6d1d4362 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:14:33.986833 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03495ca-652c-45f6-a820-06fd6d1d4362 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:14:33.995819 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:14:34.006829 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-f03495ca-652c-45f6-a820-06fd6d1d4362 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:14:34.008070 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-f03495ca-652c-45f6-a820-06fd6d1d4362 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:14:34.009982 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-91b5513a-5660-486a-97ed-e4860321bc49 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:14:34.011152 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-91b5513a-5660-486a-97ed-e4860321bc49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:14:34.015998 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 776/1557] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:14:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:14:34.017285 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d601b5-3bf3-4bef-b26a-019d5fc81822 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:14:34.017591 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d601b5-3bf3-4bef-b26a-019d5fc81822 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:14:34.017866 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d601b5-3bf3-4bef-b26a-019d5fc81822 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:14:34.021065 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 782/1558] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:14:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:14:34.021830 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b5513a-5660-486a-97ed-e4860321bc49 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:14:34.032330 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b5513a-5660-486a-97ed-e4860321bc49 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:14:34.037563 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6181b2-2e41-4545-84fe-703798f43840 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:14:34.038497 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6181b2-2e41-4545-84fe-703798f43840 admin admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:14:34.038782 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6181b2-2e41-4545-84fe-703798f43840 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:14:34.039060 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6181b2-2e41-4545-84fe-703798f43840 admin admin] PATH_INFO: `/v3/projects/6ceed8120f864c4aa210ae72a2b1e9c5/users/b7c740c7ce2e4b3c984247b6b2b69da1/roles/472313ae69174578824df0bed6390744` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:14:34.040588 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-72d601b5-3bf3-4bef-b26a-019d5fc81822 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:14:34.042409 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-72d601b5-3bf3-4bef-b26a-019d5fc81822 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:14:34.050553 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 777/1559] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:14:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:14:34.054457 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6181b2-2e41-4545-84fe-703798f43840 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6ceed8120f864c4aa210ae72a2b1e9c5, user_id=b7c740c7ce2e4b3c984247b6b2b69da1, role_id=472313ae69174578824df0bed6390744)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:14:34.054985 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d601b5-3bf3-4bef-b26a-019d5fc81822 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:14:34.055493 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: WARNING py.warnings [None req-0c6181b2-2e41-4545-84fe-703798f43840 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 25 12:14:34.055493 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: warnings.warn(msg) Jul 25 12:14:34.055493 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: Jul 25 12:14:34.058095 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6181b2-2e41-4545-84fe-703798f43840 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:14:34.065527 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d601b5-3bf3-4bef-b26a-019d5fc81822 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:14:34.067289 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:14:34.072764 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 783/1560] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Jul 25 12:14:34 2022] PUT /identity/v3/projects/6ceed8120f864c4aa210ae72a2b1e9c5/users/b7c740c7ce2e4b3c984247b6b2b69da1/roles/472313ae69174578824df0bed6390744 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:14:34.074307 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b1312f-5822-41ea-a0c8-86aee5bcc62d None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:14:34.074948 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b1312f-5822-41ea-a0c8-86aee5bcc62d None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:14:34.075424 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b1312f-5822-41ea-a0c8-86aee5bcc62d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:14:34.094629 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-b5b1312f-5822-41ea-a0c8-86aee5bcc62d None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:14:34.095977 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-b5b1312f-5822-41ea-a0c8-86aee5bcc62d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:14:34.132072 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45cdba6-2f25-4226-bb5e-3c47bc27c7ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:14:34.133151 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45cdba6-2f25-4226-bb5e-3c47bc27c7ea admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:14:34.133442 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45cdba6-2f25-4226-bb5e-3c47bc27c7ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:14:34.133722 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45cdba6-2f25-4226-bb5e-3c47bc27c7ea admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:14:34.134481 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45cdba6-2f25-4226-bb5e-3c47bc27c7ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:14:34.136238 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45cdba6-2f25-4226-bb5e-3c47bc27c7ea admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:14:34.149195 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 778/1561] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Jul 25 12:14:34 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:14:34.149195 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540b8f53-fa69-411e-8b16-eaba66aacd51 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:14:34.149195 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540b8f53-fa69-411e-8b16-eaba66aacd51 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:14:34.149195 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540b8f53-fa69-411e-8b16-eaba66aacd51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:14:34.155253 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 784/1562] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:14:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:14:34.156580 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422c4fd1-71b1-48f1-a9f8-0ec86691da6b None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:14:34.156879 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422c4fd1-71b1-48f1-a9f8-0ec86691da6b None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:14:34.157170 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422c4fd1-71b1-48f1-a9f8-0ec86691da6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:14:34.168958 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-540b8f53-fa69-411e-8b16-eaba66aacd51 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:14:34.170179 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-540b8f53-fa69-411e-8b16-eaba66aacd51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:14:34.176774 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-422c4fd1-71b1-48f1-a9f8-0ec86691da6b None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:14:34.177965 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-422c4fd1-71b1-48f1-a9f8-0ec86691da6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:14:34.178214 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 779/1563] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:14:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:14:34.179058 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540b8f53-fa69-411e-8b16-eaba66aacd51 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:14:34.187580 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 785/1564] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:14:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:14:34.188841 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a809d03-6955-496f-96d1-039aaea78984 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:14:34.189144 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a809d03-6955-496f-96d1-039aaea78984 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:14:34.189432 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a809d03-6955-496f-96d1-039aaea78984 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:14:34.189932 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540b8f53-fa69-411e-8b16-eaba66aacd51 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:14:34.194944 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d207b8-9e73-40f7-ab8e-b05b4b496e73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:14:34.195632 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d207b8-9e73-40f7-ab8e-b05b4b496e73 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:14:34.195922 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d207b8-9e73-40f7-ab8e-b05b4b496e73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:14:34.196205 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d207b8-9e73-40f7-ab8e-b05b4b496e73 admin admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:14:34.196939 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d207b8-9e73-40f7-ab8e-b05b4b496e73 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:14:34.197311 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d207b8-9e73-40f7-ab8e-b05b4b496e73 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:14:34.198888 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d207b8-9e73-40f7-ab8e-b05b4b496e73 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:14:34.208499 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-9a809d03-6955-496f-96d1-039aaea78984 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:14:34.209726 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-9a809d03-6955-496f-96d1-039aaea78984 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:14:34.211510 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 780/1565] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Jul 25 12:14:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:14:34.212418 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d207b8-9e73-40f7-ab8e-b05b4b496e73 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:14:34.222187 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d207b8-9e73-40f7-ab8e-b05b4b496e73 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:14:34.225641 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 786/1566] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:14:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:14:34.226806 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e132c05-fa44-41ff-94e1-4bed2a019d76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:14:34.226950 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b60861-2287-4d50-8124-b796fcee0cd8 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:14:34.227189 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b60861-2287-4d50-8124-b796fcee0cd8 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:14:34.227457 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e132c05-fa44-41ff-94e1-4bed2a019d76 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:14:34.227668 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b60861-2287-4d50-8124-b796fcee0cd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:14:34.227777 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e132c05-fa44-41ff-94e1-4bed2a019d76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:14:34.228041 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e132c05-fa44-41ff-94e1-4bed2a019d76 admin admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:14:34.229306 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e132c05-fa44-41ff-94e1-4bed2a019d76 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:14:34.229851 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e132c05-fa44-41ff-94e1-4bed2a019d76 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:14:34.231449 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e132c05-fa44-41ff-94e1-4bed2a019d76 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:14:34.243512 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 781/1567] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Jul 25 12:14:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:14:34.245036 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e132c05-fa44-41ff-94e1-4bed2a019d76 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:14:34.254416 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e132c05-fa44-41ff-94e1-4bed2a019d76 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:14:34.258995 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753750be-f45f-4a79-852f-a7325cf3e4fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:14:34.259692 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753750be-f45f-4a79-852f-a7325cf3e4fa admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:14:34.259984 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753750be-f45f-4a79-852f-a7325cf3e4fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:14:34.260269 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753750be-f45f-4a79-852f-a7325cf3e4fa admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:14:34.261012 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753750be-f45f-4a79-852f-a7325cf3e4fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:14:34.262595 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753750be-f45f-4a79-852f-a7325cf3e4fa admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:14:34.264563 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-33b60861-2287-4d50-8124-b796fcee0cd8 None None] MFA Rules not processed for user `b7c740c7ce2e4b3c984247b6b2b69da1`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:14:34.265776 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-33b60861-2287-4d50-8124-b796fcee0cd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:14:34.272953 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 782/1568] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Jul 25 12:14:34 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:14:34.276942 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753750be-f45f-4a79-852f-a7325cf3e4fa admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:14:34.286763 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753750be-f45f-4a79-852f-a7325cf3e4fa admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:14:34.292024 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8ad455-3422-4f15-9b1f-1aa9e2c14912 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:14:34.292674 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8ad455-3422-4f15-9b1f-1aa9e2c14912 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:14:34.292961 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8ad455-3422-4f15-9b1f-1aa9e2c14912 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:14:34.293246 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8ad455-3422-4f15-9b1f-1aa9e2c14912 admin admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:14:34.293965 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8ad455-3422-4f15-9b1f-1aa9e2c14912 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:14:34.294275 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8ad455-3422-4f15-9b1f-1aa9e2c14912 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:14:34.295866 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8ad455-3422-4f15-9b1f-1aa9e2c14912 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:14:34.309262 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 783/1569] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Jul 25 12:14:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:14:34.310051 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8ad455-3422-4f15-9b1f-1aa9e2c14912 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:14:34.320232 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8ad455-3422-4f15-9b1f-1aa9e2c14912 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:14:34.325075 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f4ece7-fd86-44db-b244-825aa6a6ee55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:14:34.325700 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f4ece7-fd86-44db-b244-825aa6a6ee55 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:14:34.326094 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f4ece7-fd86-44db-b244-825aa6a6ee55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:14:34.326382 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f4ece7-fd86-44db-b244-825aa6a6ee55 admin admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:14:34.327134 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f4ece7-fd86-44db-b244-825aa6a6ee55 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:14:34.327742 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f4ece7-fd86-44db-b244-825aa6a6ee55 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:14:34.329029 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f4ece7-fd86-44db-b244-825aa6a6ee55 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:14:34.336384 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 787/1570] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:14:34 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:14:34.340085 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b60861-2287-4d50-8124-b796fcee0cd8 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:14:34.343119 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 784/1571] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Jul 25 12:14:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:14:34.343119 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f4ece7-fd86-44db-b244-825aa6a6ee55 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:14:34.354056 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b60861-2287-4d50-8124-b796fcee0cd8 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:14:34.359617 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9d9e1c-c0ff-438a-9b6d-6a3019a455fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:14:34.360367 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9d9e1c-c0ff-438a-9b6d-6a3019a455fc admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:14:34.360725 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9d9e1c-c0ff-438a-9b6d-6a3019a455fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:14:34.360835 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f4ece7-fd86-44db-b244-825aa6a6ee55 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:14:34.361147 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9d9e1c-c0ff-438a-9b6d-6a3019a455fc admin admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:14:34.361955 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9d9e1c-c0ff-438a-9b6d-6a3019a455fc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:14:34.362269 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9d9e1c-c0ff-438a-9b6d-6a3019a455fc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:14:34.363924 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9d9e1c-c0ff-438a-9b6d-6a3019a455fc admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:14:34.366591 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81a2c50-1292-4372-a5f3-eaefd8ec4105 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:14:34.367326 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81a2c50-1292-4372-a5f3-eaefd8ec4105 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:14:34.367729 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81a2c50-1292-4372-a5f3-eaefd8ec4105 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:14:34.368114 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81a2c50-1292-4372-a5f3-eaefd8ec4105 admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:14:34.368840 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81a2c50-1292-4372-a5f3-eaefd8ec4105 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:14:34.370874 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81a2c50-1292-4372-a5f3-eaefd8ec4105 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:14:34.377765 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 788/1572] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Jul 25 12:14:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:14:34.378666 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9d9e1c-c0ff-438a-9b6d-6a3019a455fc admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:14:34.393547 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9d9e1c-c0ff-438a-9b6d-6a3019a455fc admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:14:34.398259 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c0f2e2-2447-4294-8465-73c29942a8dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:14:34.399643 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c0f2e2-2447-4294-8465-73c29942a8dd admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:14:34.399803 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 785/1573] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Jul 25 12:14:34 2022] POST /identity/v3/projects => generated 375 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:14:34.399984 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c0f2e2-2447-4294-8465-73c29942a8dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:14:34.400730 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c0f2e2-2447-4294-8465-73c29942a8dd admin admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:14:34.401011 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81a2c50-1292-4372-a5f3-eaefd8ec4105 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:14:34.401531 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c0f2e2-2447-4294-8465-73c29942a8dd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:14:34.401967 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c0f2e2-2447-4294-8465-73c29942a8dd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:14:34.403703 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c0f2e2-2447-4294-8465-73c29942a8dd admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:14:34.417548 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 789/1574] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Jul 25 12:14:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:14:34.418402 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c0f2e2-2447-4294-8465-73c29942a8dd admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:14:34.431490 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c0f2e2-2447-4294-8465-73c29942a8dd admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:14:34.458752 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:14:34.465049 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81a2c50-1292-4372-a5f3-eaefd8ec4105 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:14:34.470809 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8000320b-4834-4767-8a11-50e39a9cf2a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:14:34.471476 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8000320b-4834-4767-8a11-50e39a9cf2a2 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:14:34.471796 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8000320b-4834-4767-8a11-50e39a9cf2a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:14:34.472084 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8000320b-4834-4767-8a11-50e39a9cf2a2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:14:34.472778 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8000320b-4834-4767-8a11-50e39a9cf2a2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:14:34.474736 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8000320b-4834-4767-8a11-50e39a9cf2a2 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:14:34.477271 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:14:34.489812 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7194ce3c-deed-4111-b15d-3f79e52adfdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:14:34.490705 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7194ce3c-deed-4111-b15d-3f79e52adfdb admin admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:14:34.490991 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7194ce3c-deed-4111-b15d-3f79e52adfdb admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:14:34.491377 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7194ce3c-deed-4111-b15d-3f79e52adfdb admin admin] PATH_INFO: `/v3/projects/aef3a5bd8ffa43718ea02647f1f74fc8/users/d2d6f10a6fe841e5b1d12bce78f97d91/roles/472313ae69174578824df0bed6390744` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:14:34.502298 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 786/1575] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Jul 25 12:14:34 2022] POST /identity/v3/projects => generated 371 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:14:34.503369 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8000320b-4834-4767-8a11-50e39a9cf2a2 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:14:34.508570 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7194ce3c-deed-4111-b15d-3f79e52adfdb admin admin] RBAC: Authorizing `identity:create_grant(project_id=aef3a5bd8ffa43718ea02647f1f74fc8, user_id=d2d6f10a6fe841e5b1d12bce78f97d91, role_id=472313ae69174578824df0bed6390744)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:14:34.511301 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7194ce3c-deed-4111-b15d-3f79e52adfdb admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:14:34.523978 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 790/1576] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Jul 25 12:14:34 2022] PUT /identity/v3/projects/aef3a5bd8ffa43718ea02647f1f74fc8/users/d2d6f10a6fe841e5b1d12bce78f97d91/roles/472313ae69174578824df0bed6390744 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:14:34.528309 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7194ce3c-deed-4111-b15d-3f79e52adfdb admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:14:34.569695 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8000320b-4834-4767-8a11-50e39a9cf2a2 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:14:34.586271 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7194ce3c-deed-4111-b15d-3f79e52adfdb admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:14:34.591986 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f57c14-a149-493b-9ab5-fc29e4f492a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:14:34.592651 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f57c14-a149-493b-9ab5-fc29e4f492a9 admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:14:34.592941 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f57c14-a149-493b-9ab5-fc29e4f492a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:14:34.593224 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f57c14-a149-493b-9ab5-fc29e4f492a9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:14:34.593920 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f57c14-a149-493b-9ab5-fc29e4f492a9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:14:34.595857 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f57c14-a149-493b-9ab5-fc29e4f492a9 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:14:34.623605 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 791/1577] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Jul 25 12:14:34 2022] POST /identity/v3/projects => generated 363 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:14:34.624912 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e93b062-5603-4f37-9018-bdc5f7485c9f None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:14:34.625206 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e93b062-5603-4f37-9018-bdc5f7485c9f None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:14:34.625494 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e93b062-5603-4f37-9018-bdc5f7485c9f None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:14:34.626919 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 792/1578] 10.222.0.42 () {54 vars in 962 bytes} [Mon Jul 25 12:14:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:14:34.627509 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e93b062-5603-4f37-9018-bdc5f7485c9f None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:14:34.631159 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db326a61-dd14-49dd-8e62-07c955434a9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:14:34.631877 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db326a61-dd14-49dd-8e62-07c955434a9b admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:14:34.632412 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db326a61-dd14-49dd-8e62-07c955434a9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:14:34.632699 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db326a61-dd14-49dd-8e62-07c955434a9b admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:14:34.633402 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db326a61-dd14-49dd-8e62-07c955434a9b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:14:34.635379 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db326a61-dd14-49dd-8e62-07c955434a9b admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:14:34.662888 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 787/1579] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Jul 25 12:14:34 2022] POST /identity/v3/projects => generated 393 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:14:34.667449 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db326a61-dd14-49dd-8e62-07c955434a9b admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:14:34.692895 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e93b062-5603-4f37-9018-bdc5f7485c9f None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:14:34.727880 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db326a61-dd14-49dd-8e62-07c955434a9b admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:14:34.732508 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4e0f77-d265-4162-b2b2-ad7958e645b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:14:34.733177 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4e0f77-d265-4162-b2b2-ad7958e645b6 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:14:34.733471 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4e0f77-d265-4162-b2b2-ad7958e645b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:14:34.733758 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4e0f77-d265-4162-b2b2-ad7958e645b6 admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:14:34.734453 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4e0f77-d265-4162-b2b2-ad7958e645b6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:14:34.736457 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4e0f77-d265-4162-b2b2-ad7958e645b6 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:14:34.750797 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097d3de9-abfa-4147-846f-63cd59df8b2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:14:34.751465 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097d3de9-abfa-4147-846f-63cd59df8b2a admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:14:34.751853 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097d3de9-abfa-4147-846f-63cd59df8b2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:14:34.752142 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097d3de9-abfa-4147-846f-63cd59df8b2a admin admin] PATH_INFO: `/v3/projects` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:14:34.753159 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097d3de9-abfa-4147-846f-63cd59df8b2a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:14:34.755064 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097d3de9-abfa-4147-846f-63cd59df8b2a admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:14:34.758063 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 788/1580] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Jul 25 12:14:34 2022] POST /identity/v3/users => generated 288 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:14:34.762613 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4e0f77-d265-4162-b2b2-ad7958e645b6 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:14:34.765081 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:14:34.767614 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:14:34.776924 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4e0f77-d265-4162-b2b2-ad7958e645b6 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:14:34.783076 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75be569-9355-4202-8332-879e01446cbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:14:34.783765 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75be569-9355-4202-8332-879e01446cbe admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:14:34.784057 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75be569-9355-4202-8332-879e01446cbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:14:34.784374 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75be569-9355-4202-8332-879e01446cbe admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:14:34.785033 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75be569-9355-4202-8332-879e01446cbe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:14:34.787193 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75be569-9355-4202-8332-879e01446cbe admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:14:34.792054 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 793/1581] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Jul 25 12:14:34 2022] POST /identity/v3/projects => generated 385 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:14:34.792995 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097d3de9-abfa-4147-846f-63cd59df8b2a admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:14:34.821076 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 789/1582] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Jul 25 12:14:34 2022] POST /identity/v3/projects => generated 379 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:14:34.821076 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d93295-4989-4fbd-af2a-c9e6f833507b None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:14:34.821076 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d93295-4989-4fbd-af2a-c9e6f833507b None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:14:34.821076 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d93295-4989-4fbd-af2a-c9e6f833507b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:14:34.840222 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-97d93295-4989-4fbd-af2a-c9e6f833507b None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:14:34.841430 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-97d93295-4989-4fbd-af2a-c9e6f833507b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:14:34.859597 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097d3de9-abfa-4147-846f-63cd59df8b2a admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:14:34.899770 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 790/1583] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:14:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:14:34.904282 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d93295-4989-4fbd-af2a-c9e6f833507b None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:14:34.913384 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d93295-4989-4fbd-af2a-c9e6f833507b None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:14:34.916461 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c85e26-a4f4-4cb5-a3f6-a117cd2d8af4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:14:34.917133 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c85e26-a4f4-4cb5-a3f6-a117cd2d8af4 admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:14:34.917419 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c85e26-a4f4-4cb5-a3f6-a117cd2d8af4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:14:34.917697 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c85e26-a4f4-4cb5-a3f6-a117cd2d8af4 admin admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:14:34.918386 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c85e26-a4f4-4cb5-a3f6-a117cd2d8af4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:14:34.918745 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4265707d-12c5-4d74-9ae1-4f15f07ed951 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:14:34.919409 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4265707d-12c5-4d74-9ae1-4f15f07ed951 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:14:34.919745 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4265707d-12c5-4d74-9ae1-4f15f07ed951 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:14:34.920032 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4265707d-12c5-4d74-9ae1-4f15f07ed951 admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:14:34.920361 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c85e26-a4f4-4cb5-a3f6-a117cd2d8af4 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:14:34.920730 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4265707d-12c5-4d74-9ae1-4f15f07ed951 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:14:34.922654 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4265707d-12c5-4d74-9ae1-4f15f07ed951 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:14:34.947976 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 794/1584] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Jul 25 12:14:34 2022] POST /identity/v3/users => generated 286 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:14:34.948262 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 791/1585] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Jul 25 12:14:34 2022] POST /identity/v3/users => generated 282 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:14:34.948262 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4265707d-12c5-4d74-9ae1-4f15f07ed951 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:14:34.948455 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdac9e25-b8df-48cc-8411-5a760dd00445 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:14:34.948603 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdac9e25-b8df-48cc-8411-5a760dd00445 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:14:34.948918 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdac9e25-b8df-48cc-8411-5a760dd00445 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:14:34.957514 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4265707d-12c5-4d74-9ae1-4f15f07ed951 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:14:34.962039 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20d4249-9261-4abe-9db0-35e5cb39ba28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:14:34.962772 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20d4249-9261-4abe-9db0-35e5cb39ba28 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:14:34.963027 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20d4249-9261-4abe-9db0-35e5cb39ba28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:14:34.963309 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20d4249-9261-4abe-9db0-35e5cb39ba28 admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:14:34.964305 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20d4249-9261-4abe-9db0-35e5cb39ba28 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:14:34.966225 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20d4249-9261-4abe-9db0-35e5cb39ba28 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:14:34.990265 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 792/1586] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Jul 25 12:14:34 2022] POST /identity/v3/users => generated 297 bytes in 42 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:14:34.990265 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20d4249-9261-4abe-9db0-35e5cb39ba28 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:14:34.991566 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-fdac9e25-b8df-48cc-8411-5a760dd00445 None None] MFA Rules not processed for user `f67142a1193d47c7bd33d7040268233c`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:14:34.992850 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-fdac9e25-b8df-48cc-8411-5a760dd00445 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:14:34.998459 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20d4249-9261-4abe-9db0-35e5cb39ba28 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:14:35.003135 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28852d2-0fe3-4aaf-846a-faf7460dfe34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:14:35.003817 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28852d2-0fe3-4aaf-846a-faf7460dfe34 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:14:35.004110 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28852d2-0fe3-4aaf-846a-faf7460dfe34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:14:35.004395 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28852d2-0fe3-4aaf-846a-faf7460dfe34 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:14:35.005122 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28852d2-0fe3-4aaf-846a-faf7460dfe34 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:14:35.007043 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28852d2-0fe3-4aaf-846a-faf7460dfe34 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:14:35.017447 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 793/1587] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Jul 25 12:14:34 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:14:35.018198 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28852d2-0fe3-4aaf-846a-faf7460dfe34 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:14:35.027366 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28852d2-0fe3-4aaf-846a-faf7460dfe34 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:14:35.031976 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d777e01f-c3c4-46bf-9711-10691e69ebc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:14:35.032606 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d777e01f-c3c4-46bf-9711-10691e69ebc6 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:14:35.032897 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d777e01f-c3c4-46bf-9711-10691e69ebc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:14:35.033181 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d777e01f-c3c4-46bf-9711-10691e69ebc6 admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:14:35.033861 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d777e01f-c3c4-46bf-9711-10691e69ebc6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:14:35.036054 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d777e01f-c3c4-46bf-9711-10691e69ebc6 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:14:35.058580 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 794/1588] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Jul 25 12:14:35 2022] POST /identity/v3/users => generated 293 bytes in 41 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:14:35.059741 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d777e01f-c3c4-46bf-9711-10691e69ebc6 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:14:35.065891 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 795/1589] 10.222.0.42 () {58 vars in 1059 bytes} [Mon Jul 25 12:14:34 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:14:35.066729 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdac9e25-b8df-48cc-8411-5a760dd00445 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:14:35.070727 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d777e01f-c3c4-46bf-9711-10691e69ebc6 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:14:35.072105 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:14:35.075990 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d7b979-798b-4215-b647-7187e24cb67f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:14:35.076650 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d7b979-798b-4215-b647-7187e24cb67f admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:14:35.076966 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d7b979-798b-4215-b647-7187e24cb67f admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:14:35.077248 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d7b979-798b-4215-b647-7187e24cb67f admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:14:35.077939 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d7b979-798b-4215-b647-7187e24cb67f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:14:35.079932 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d7b979-798b-4215-b647-7187e24cb67f admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:14:35.101954 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 795/1590] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Jul 25 12:14:35 2022] POST /identity/v3/users => generated 290 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:14:35.103095 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d7b979-798b-4215-b647-7187e24cb67f admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:14:35.113312 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d7b979-798b-4215-b647-7187e24cb67f admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:14:35.118079 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9888529-7a53-4a6f-b28e-7ce881722d69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:14:35.119089 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9888529-7a53-4a6f-b28e-7ce881722d69 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:14:35.119414 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9888529-7a53-4a6f-b28e-7ce881722d69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:14:35.119733 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9888529-7a53-4a6f-b28e-7ce881722d69 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:14:35.120463 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9888529-7a53-4a6f-b28e-7ce881722d69 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:14:35.122068 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9888529-7a53-4a6f-b28e-7ce881722d69 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:14:35.128273 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdac9e25-b8df-48cc-8411-5a760dd00445 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:14:35.131942 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b390c967-2e6a-4b27-9786-9a42512023e9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51733781201243e8abf9eca39b207750', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:14:35.132385 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 796/1591] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Jul 25 12:14:35 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:14:35.132610 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b390c967-2e6a-4b27-9786-9a42512023e9 service neutron] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:14:35.132879 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b390c967-2e6a-4b27-9786-9a42512023e9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:14:35.133177 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9888529-7a53-4a6f-b28e-7ce881722d69 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:14:35.133280 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b390c967-2e6a-4b27-9786-9a42512023e9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:14:35.143198 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b390c967-2e6a-4b27-9786-9a42512023e9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:14:35.144703 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b390c967-2e6a-4b27-9786-9a42512023e9 service neutron] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:14:35.145376 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9888529-7a53-4a6f-b28e-7ce881722d69 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:14:35.150947 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac58de0d-1aeb-4c79-9a7a-9bd5b8f30940 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:14:35.151604 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac58de0d-1aeb-4c79-9a7a-9bd5b8f30940 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:14:35.151891 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac58de0d-1aeb-4c79-9a7a-9bd5b8f30940 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:14:35.152169 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac58de0d-1aeb-4c79-9a7a-9bd5b8f30940 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:14:35.152880 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac58de0d-1aeb-4c79-9a7a-9bd5b8f30940 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:14:35.154428 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac58de0d-1aeb-4c79-9a7a-9bd5b8f30940 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:14:35.154950 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 796/1592] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:14:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:14:35.156025 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b390c967-2e6a-4b27-9786-9a42512023e9 service neutron] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:14:35.165147 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 797/1593] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Jul 25 12:14:35 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:14:35.166647 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac58de0d-1aeb-4c79-9a7a-9bd5b8f30940 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:14:35.167929 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b390c967-2e6a-4b27-9786-9a42512023e9 service neutron] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:14:35.173356 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bbffb5-b0aa-4b38-9e14-d93ee87d5da3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:14:35.175561 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bbffb5-b0aa-4b38-9e14-d93ee87d5da3 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:14:35.175863 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bbffb5-b0aa-4b38-9e14-d93ee87d5da3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:14:35.176175 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bbffb5-b0aa-4b38-9e14-d93ee87d5da3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:14:35.176954 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bbffb5-b0aa-4b38-9e14-d93ee87d5da3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:14:35.177563 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac58de0d-1aeb-4c79-9a7a-9bd5b8f30940 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:14:35.178565 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bbffb5-b0aa-4b38-9e14-d93ee87d5da3 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:14:35.183279 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65291f9b-3ce4-46ec-83f2-4ac512e0ebb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:14:35.184022 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65291f9b-3ce4-46ec-83f2-4ac512e0ebb9 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:14:35.184407 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65291f9b-3ce4-46ec-83f2-4ac512e0ebb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:14:35.184655 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65291f9b-3ce4-46ec-83f2-4ac512e0ebb9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:14:35.185931 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65291f9b-3ce4-46ec-83f2-4ac512e0ebb9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:14:35.187617 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65291f9b-3ce4-46ec-83f2-4ac512e0ebb9 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:14:35.194066 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 797/1594] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Jul 25 12:14:35 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:14:35.194066 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bbffb5-b0aa-4b38-9e14-d93ee87d5da3 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:14:35.200371 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 798/1595] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Jul 25 12:14:35 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:14:35.204218 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65291f9b-3ce4-46ec-83f2-4ac512e0ebb9 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:14:35.204968 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bbffb5-b0aa-4b38-9e14-d93ee87d5da3 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:14:35.210005 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be241875-8289-48d6-b91f-d00370438c66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:14:35.210759 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be241875-8289-48d6-b91f-d00370438c66 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:14:35.211206 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be241875-8289-48d6-b91f-d00370438c66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:14:35.211796 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be241875-8289-48d6-b91f-d00370438c66 admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:14:35.213868 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be241875-8289-48d6-b91f-d00370438c66 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:14:35.215980 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be241875-8289-48d6-b91f-d00370438c66 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:14:35.218745 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65291f9b-3ce4-46ec-83f2-4ac512e0ebb9 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:14:35.224712 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363cf7c6-1e50-483a-a55f-5b5a401ce268 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f67142a1193d47c7bd33d7040268233c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:14:35.225364 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363cf7c6-1e50-483a-a55f-5b5a401ce268 service nova] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:14:35.225658 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363cf7c6-1e50-483a-a55f-5b5a401ce268 service nova] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:14:35.225961 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363cf7c6-1e50-483a-a55f-5b5a401ce268 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:14:35.233043 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 798/1596] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Jul 25 12:14:35 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:14:35.233043 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be241875-8289-48d6-b91f-d00370438c66 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:14:35.244439 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be241875-8289-48d6-b91f-d00370438c66 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:14:35.250322 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a262a6-1b5e-47ea-a32a-83b5fe733597 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:14:35.251318 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a262a6-1b5e-47ea-a32a-83b5fe733597 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:14:35.251677 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a262a6-1b5e-47ea-a32a-83b5fe733597 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:14:35.252012 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a262a6-1b5e-47ea-a32a-83b5fe733597 admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:14:35.252751 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a262a6-1b5e-47ea-a32a-83b5fe733597 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:14:35.254623 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a262a6-1b5e-47ea-a32a-83b5fe733597 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:14:35.263167 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=87,keystone:INSERT=13 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:14:35.269471 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 799/1597] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Jul 25 12:14:35 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:14:35.273796 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a262a6-1b5e-47ea-a32a-83b5fe733597 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:14:35.286576 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a262a6-1b5e-47ea-a32a-83b5fe733597 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:14:35.292142 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280e857d-11fe-49b0-b83a-21b555b2aa02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:14:35.292949 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280e857d-11fe-49b0-b83a-21b555b2aa02 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:14:35.293246 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280e857d-11fe-49b0-b83a-21b555b2aa02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:14:35.293526 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280e857d-11fe-49b0-b83a-21b555b2aa02 admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:14:35.294256 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280e857d-11fe-49b0-b83a-21b555b2aa02 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:14:35.295905 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280e857d-11fe-49b0-b83a-21b555b2aa02 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:14:35.302872 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363cf7c6-1e50-483a-a55f-5b5a401ce268 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:14:35.304667 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363cf7c6-1e50-483a-a55f-5b5a401ce268 service nova] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:14:35.308078 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 800/1598] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Jul 25 12:14:35 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:14:35.309053 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280e857d-11fe-49b0-b83a-21b555b2aa02 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:14:35.318593 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 799/1599] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:14:35 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:14:35.319372 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363cf7c6-1e50-483a-a55f-5b5a401ce268 service nova] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:14:35.326257 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280e857d-11fe-49b0-b83a-21b555b2aa02 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:14:35.331081 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363cf7c6-1e50-483a-a55f-5b5a401ce268 service nova] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:14:35.333691 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7318a90-e416-470e-8b5c-441563697d01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:14:35.334781 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7318a90-e416-470e-8b5c-441563697d01 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:14:35.335264 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7318a90-e416-470e-8b5c-441563697d01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:14:35.335794 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7318a90-e416-470e-8b5c-441563697d01 admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:14:35.336853 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131b9e88-b5f0-444e-b27b-28b6874e4e2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:14:35.337153 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7318a90-e416-470e-8b5c-441563697d01 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:14:35.338199 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131b9e88-b5f0-444e-b27b-28b6874e4e2f admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:14:35.338583 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131b9e88-b5f0-444e-b27b-28b6874e4e2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:14:35.338874 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131b9e88-b5f0-444e-b27b-28b6874e4e2f admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:14:35.339689 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131b9e88-b5f0-444e-b27b-28b6874e4e2f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:14:35.340922 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7318a90-e416-470e-8b5c-441563697d01 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:14:35.341339 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131b9e88-b5f0-444e-b27b-28b6874e4e2f admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:14:35.359588 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 800/1600] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Jul 25 12:14:35 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:14:35.359588 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131b9e88-b5f0-444e-b27b-28b6874e4e2f admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:14:35.360498 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 801/1601] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Jul 25 12:14:35 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:14:35.361458 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7318a90-e416-470e-8b5c-441563697d01 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:14:35.370371 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131b9e88-b5f0-444e-b27b-28b6874e4e2f admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:14:35.376047 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f658c2fa-6166-46bf-ab20-6e891cb01818 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:14:35.376426 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7318a90-e416-470e-8b5c-441563697d01 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:14:35.377112 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f658c2fa-6166-46bf-ab20-6e891cb01818 admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:14:35.377935 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f658c2fa-6166-46bf-ab20-6e891cb01818 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:14:35.378291 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f658c2fa-6166-46bf-ab20-6e891cb01818 admin admin] PATH_INFO: `/v3/projects/ce78c16fa24c475b945a64a7995c70e4/users/c5b98bbcc2d640de837d141f83dfe924/roles/472313ae69174578824df0bed6390744` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:14:35.383713 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcd3851-1d89-41c3-9237-a39f0ebfe397 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:14:35.384406 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcd3851-1d89-41c3-9237-a39f0ebfe397 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:14:35.384812 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcd3851-1d89-41c3-9237-a39f0ebfe397 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:14:35.385187 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcd3851-1d89-41c3-9237-a39f0ebfe397 admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:14:35.386015 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fcd3851-1d89-41c3-9237-a39f0ebfe397 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:14:35.388279 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fcd3851-1d89-41c3-9237-a39f0ebfe397 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:14:35.399331 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f658c2fa-6166-46bf-ab20-6e891cb01818 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce78c16fa24c475b945a64a7995c70e4, user_id=c5b98bbcc2d640de837d141f83dfe924, role_id=472313ae69174578824df0bed6390744)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:14:35.402767 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f658c2fa-6166-46bf-ab20-6e891cb01818 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:14:35.403785 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 802/1602] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Jul 25 12:14:35 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:14:35.405369 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcd3851-1d89-41c3-9237-a39f0ebfe397 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:14:35.417698 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcd3851-1d89-41c3-9237-a39f0ebfe397 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:14:35.422765 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 801/1603] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Jul 25 12:14:35 2022] PUT /identity/v3/projects/ce78c16fa24c475b945a64a7995c70e4/users/c5b98bbcc2d640de837d141f83dfe924/roles/472313ae69174578824df0bed6390744 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:14:35.423768 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f658c2fa-6166-46bf-ab20-6e891cb01818 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:14:35.492835 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca502d01-bcd4-4b84-909b-d2bf4a1122eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:14:35.493802 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca502d01-bcd4-4b84-909b-d2bf4a1122eb admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:14:35.494578 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca502d01-bcd4-4b84-909b-d2bf4a1122eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:14:35.494889 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca502d01-bcd4-4b84-909b-d2bf4a1122eb admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:14:35.495737 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca502d01-bcd4-4b84-909b-d2bf4a1122eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:14:35.497468 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca502d01-bcd4-4b84-909b-d2bf4a1122eb admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:14:35.509441 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 803/1604] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Jul 25 12:14:35 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:14:35.510904 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca502d01-bcd4-4b84-909b-d2bf4a1122eb admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:14:35.514415 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f658c2fa-6166-46bf-ab20-6e891cb01818 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:14:35.520000 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782d35f2-36f0-44e1-ac4b-2c0cbc9f6354 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:14:35.521698 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782d35f2-36f0-44e1-ac4b-2c0cbc9f6354 admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:14:35.522381 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782d35f2-36f0-44e1-ac4b-2c0cbc9f6354 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:14:35.522613 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca502d01-bcd4-4b84-909b-d2bf4a1122eb admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:14:35.522992 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782d35f2-36f0-44e1-ac4b-2c0cbc9f6354 admin admin] PATH_INFO: `/v3/projects/13ebc33acf09437699e79e3ba39e14bb/users/880ee36229b1452f84928299f8b8926e/roles/472313ae69174578824df0bed6390744` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:14:35.528400 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b511aacb-bdcf-4f13-8d9d-a649bc9dd785 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:14:35.529359 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b511aacb-bdcf-4f13-8d9d-a649bc9dd785 admin admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:14:35.529719 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b511aacb-bdcf-4f13-8d9d-a649bc9dd785 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:14:35.530075 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b511aacb-bdcf-4f13-8d9d-a649bc9dd785 admin admin] PATH_INFO: `/v3/projects/2d9f339447114912b0c92b99e028fbbf/users/a766d17b116f498fab07efb69c94e49e/roles/472313ae69174578824df0bed6390744` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:14:35.546347 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782d35f2-36f0-44e1-ac4b-2c0cbc9f6354 admin admin] RBAC: Authorizing `identity:create_grant(project_id=13ebc33acf09437699e79e3ba39e14bb, user_id=880ee36229b1452f84928299f8b8926e, role_id=472313ae69174578824df0bed6390744)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:14:35.548261 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b511aacb-bdcf-4f13-8d9d-a649bc9dd785 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2d9f339447114912b0c92b99e028fbbf, user_id=a766d17b116f498fab07efb69c94e49e, role_id=472313ae69174578824df0bed6390744)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:14:35.550166 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782d35f2-36f0-44e1-ac4b-2c0cbc9f6354 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:14:35.551367 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b511aacb-bdcf-4f13-8d9d-a649bc9dd785 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:14:35.568346 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 802/1605] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Jul 25 12:14:35 2022] PUT /identity/v3/projects/13ebc33acf09437699e79e3ba39e14bb/users/880ee36229b1452f84928299f8b8926e/roles/472313ae69174578824df0bed6390744 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:14:35.569418 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782d35f2-36f0-44e1-ac4b-2c0cbc9f6354 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:14:35.578277 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 804/1606] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Jul 25 12:14:35 2022] PUT /identity/v3/projects/2d9f339447114912b0c92b99e028fbbf/users/a766d17b116f498fab07efb69c94e49e/roles/472313ae69174578824df0bed6390744 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:14:35.579289 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b511aacb-bdcf-4f13-8d9d-a649bc9dd785 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:14:35.608774 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:14:35.690410 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782d35f2-36f0-44e1-ac4b-2c0cbc9f6354 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:14:35.694067 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b511aacb-bdcf-4f13-8d9d-a649bc9dd785 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:14:35.702392 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152ff2b0-b7ca-4d9c-829f-d4e345ad3d9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:14:35.702601 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e256c66a-dae1-4eab-a277-4a02c93c88ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:14:35.703638 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e256c66a-dae1-4eab-a277-4a02c93c88ec admin admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:14:35.704006 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e256c66a-dae1-4eab-a277-4a02c93c88ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:14:35.704213 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152ff2b0-b7ca-4d9c-829f-d4e345ad3d9f admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:14:35.704370 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e256c66a-dae1-4eab-a277-4a02c93c88ec admin admin] PATH_INFO: `/v3/projects/11ff6cbb1ad04f0faa8b2ed2a66cdca4/users/3848eae5b8df48eb9816ba0d7485f7c0/roles/472313ae69174578824df0bed6390744` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:14:35.704977 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152ff2b0-b7ca-4d9c-829f-d4e345ad3d9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:14:35.705567 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152ff2b0-b7ca-4d9c-829f-d4e345ad3d9f admin admin] PATH_INFO: `/v3/projects/e7e349c1736b41e5ac722809adbf3c69/users/196c9e67f28046ee98a08a39d0f1e4ba/roles/472313ae69174578824df0bed6390744` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:14:35.726154 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e256c66a-dae1-4eab-a277-4a02c93c88ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=11ff6cbb1ad04f0faa8b2ed2a66cdca4, user_id=3848eae5b8df48eb9816ba0d7485f7c0, role_id=472313ae69174578824df0bed6390744)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:14:35.726582 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152ff2b0-b7ca-4d9c-829f-d4e345ad3d9f admin admin] RBAC: Authorizing `identity:create_grant(project_id=e7e349c1736b41e5ac722809adbf3c69, user_id=196c9e67f28046ee98a08a39d0f1e4ba, role_id=472313ae69174578824df0bed6390744)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:14:35.729404 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e256c66a-dae1-4eab-a277-4a02c93c88ec admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:14:35.729571 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152ff2b0-b7ca-4d9c-829f-d4e345ad3d9f admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:14:35.734220 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:14:35.746113 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 805/1607] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Jul 25 12:14:35 2022] PUT /identity/v3/projects/11ff6cbb1ad04f0faa8b2ed2a66cdca4/users/3848eae5b8df48eb9816ba0d7485f7c0/roles/472313ae69174578824df0bed6390744 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:14:35.747632 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 803/1608] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Jul 25 12:14:35 2022] PUT /identity/v3/projects/e7e349c1736b41e5ac722809adbf3c69/users/196c9e67f28046ee98a08a39d0f1e4ba/roles/472313ae69174578824df0bed6390744 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:14:35.748063 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba34dc9-5f08-49b0-a1dc-4e53edf34fbf None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:14:35.748531 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba34dc9-5f08-49b0-a1dc-4e53edf34fbf None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:14:35.748650 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152ff2b0-b7ca-4d9c-829f-d4e345ad3d9f admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:14:35.749382 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba34dc9-5f08-49b0-a1dc-4e53edf34fbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:14:35.771229 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-2ba34dc9-5f08-49b0-a1dc-4e53edf34fbf None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:14:35.772147 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-2ba34dc9-5f08-49b0-a1dc-4e53edf34fbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:14:35.832401 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152ff2b0-b7ca-4d9c-829f-d4e345ad3d9f admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:14:35.837332 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f180b8-b61c-465b-89a3-c8ccae992830 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:14:35.838518 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f180b8-b61c-465b-89a3-c8ccae992830 admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:14:35.838840 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f180b8-b61c-465b-89a3-c8ccae992830 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:14:35.839183 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f180b8-b61c-465b-89a3-c8ccae992830 admin admin] PATH_INFO: `/v3/projects/05e8cdf9803c4388a79e123894ab34b1/users/89a96a9ce9db422f96927b7f59606998/roles/472313ae69174578824df0bed6390744` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:14:35.847110 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 806/1609] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:14:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:14:35.848753 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abae620a-e8c7-46d8-ae00-786d75c0a5c1 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:14:35.849050 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abae620a-e8c7-46d8-ae00-786d75c0a5c1 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:14:35.849336 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abae620a-e8c7-46d8-ae00-786d75c0a5c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:14:35.860338 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f180b8-b61c-465b-89a3-c8ccae992830 admin admin] RBAC: Authorizing `identity:create_grant(project_id=05e8cdf9803c4388a79e123894ab34b1, user_id=89a96a9ce9db422f96927b7f59606998, role_id=472313ae69174578824df0bed6390744)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:14:35.863830 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f180b8-b61c-465b-89a3-c8ccae992830 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:14:35.879910 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-abae620a-e8c7-46d8-ae00-786d75c0a5c1 None None] MFA Rules not processed for user `880ee36229b1452f84928299f8b8926e`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:14:35.885313 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 804/1610] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Jul 25 12:14:35 2022] PUT /identity/v3/projects/05e8cdf9803c4388a79e123894ab34b1/users/89a96a9ce9db422f96927b7f59606998/roles/472313ae69174578824df0bed6390744 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:14:35.885609 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-abae620a-e8c7-46d8-ae00-786d75c0a5c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:14:35.885723 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0e79ee-384e-4653-9f03-734955012c7d None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:14:35.885723 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0e79ee-384e-4653-9f03-734955012c7d None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:14:35.885723 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0e79ee-384e-4653-9f03-734955012c7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:14:35.909795 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-8b0e79ee-384e-4653-9f03-734955012c7d None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:14:35.911287 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-8b0e79ee-384e-4653-9f03-734955012c7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:14:35.956031 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 807/1611] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:14:35 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:14:35.959793 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d610a7f5-aab2-43b0-940f-73a83c6479e0 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:14:35.959793 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d610a7f5-aab2-43b0-940f-73a83c6479e0 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:14:35.959793 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d610a7f5-aab2-43b0-940f-73a83c6479e0 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:14:35.959793 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 808/1612] 10.222.0.42 () {54 vars in 912 bytes} [Mon Jul 25 12:14:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:14:35.960468 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ef51e0-6c55-4b96-9c69-6ead073ef36d None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:14:35.961499 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ef51e0-6c55-4b96-9c69-6ead073ef36d None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:14:35.961499 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ef51e0-6c55-4b96-9c69-6ead073ef36d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:14:35.981099 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-03ef51e0-6c55-4b96-9c69-6ead073ef36d None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:14:35.982472 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-03ef51e0-6c55-4b96-9c69-6ead073ef36d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:14:35.998214 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 805/1613] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:14:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:14:35.999607 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7e22ae-f8ca-4b81-8f54-86196fb17693 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:14:35.999911 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7e22ae-f8ca-4b81-8f54-86196fb17693 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:14:36.000199 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7e22ae-f8ca-4b81-8f54-86196fb17693 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:14:36.010852 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:14:36.044374 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-dd7e22ae-f8ca-4b81-8f54-86196fb17693 None None] MFA Rules not processed for user `196c9e67f28046ee98a08a39d0f1e4ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:14:36.045961 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-dd7e22ae-f8ca-4b81-8f54-86196fb17693 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:14:36.083471 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 809/1614] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:14:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:14:36.084871 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ef51e0-6c55-4b96-9c69-6ead073ef36d None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:14:36.146863 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 806/1615] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:14:35 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:14:36.146863 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ac9808-e5f9-47e5-b8a5-26d2ae831312 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:14:36.146863 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ac9808-e5f9-47e5-b8a5-26d2ae831312 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:14:36.146863 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ac9808-e5f9-47e5-b8a5-26d2ae831312 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:14:36.166115 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ef51e0-6c55-4b96-9c69-6ead073ef36d None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:14:36.170152 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-70ac9808-e5f9-47e5-b8a5-26d2ae831312 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:14:36.170293 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5191f912-10fc-4adf-a779-d1df4d07bb5f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51733781201243e8abf9eca39b207750', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:14:36.170906 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5191f912-10fc-4adf-a779-d1df4d07bb5f service neutron] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:14:36.171111 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5191f912-10fc-4adf-a779-d1df4d07bb5f service neutron] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:14:36.171397 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5191f912-10fc-4adf-a779-d1df4d07bb5f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:14:36.171966 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-70ac9808-e5f9-47e5-b8a5-26d2ae831312 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:14:36.182760 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5191f912-10fc-4adf-a779-d1df4d07bb5f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:14:36.184226 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5191f912-10fc-4adf-a779-d1df4d07bb5f service neutron] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:14:36.194223 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 807/1616] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:14:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:14:36.195799 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ac9808-e5f9-47e5-b8a5-26d2ae831312 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:14:36.195921 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 810/1617] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:14:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:14:36.197747 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c7d92b-f2ea-4875-90bb-2ca14d90f760 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:14:36.198185 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c7d92b-f2ea-4875-90bb-2ca14d90f760 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:14:36.198469 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c7d92b-f2ea-4875-90bb-2ca14d90f760 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:14:36.202598 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:14:36.209302 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ac9808-e5f9-47e5-b8a5-26d2ae831312 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:14:36.216528 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd17a626-e8d7-4c20-817e-a20d8190d0aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:14:36.217344 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd17a626-e8d7-4c20-817e-a20d8190d0aa admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:14:36.217644 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd17a626-e8d7-4c20-817e-a20d8190d0aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:14:36.217983 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd17a626-e8d7-4c20-817e-a20d8190d0aa admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:14:36.218842 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd17a626-e8d7-4c20-817e-a20d8190d0aa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:14:36.221437 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd17a626-e8d7-4c20-817e-a20d8190d0aa admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:14:36.223004 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-32c7d92b-f2ea-4875-90bb-2ca14d90f760 None None] MFA Rules not processed for user `f67142a1193d47c7bd33d7040268233c`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:14:36.224867 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-32c7d92b-f2ea-4875-90bb-2ca14d90f760 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:14:36.257516 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 808/1618] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Jul 25 12:14:36 2022] POST /identity/v3/projects => generated 365 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:14:36.259073 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd17a626-e8d7-4c20-817e-a20d8190d0aa admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:14:36.264518 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-cd17a626-e8d7-4c20-817e-a20d8190d0aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:14:36.334450 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 811/1619] 10.222.0.42 () {58 vars in 1009 bytes} [Mon Jul 25 12:14:36 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:14:36.336191 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c7d92b-f2ea-4875-90bb-2ca14d90f760 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:14:36.346521 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd17a626-e8d7-4c20-817e-a20d8190d0aa admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:14:36.350662 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c7d92b-f2ea-4875-90bb-2ca14d90f760 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:14:36.351281 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29004d85-c077-4d9e-9f59-0fa5b98d451d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51733781201243e8abf9eca39b207750', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:14:36.352682 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29004d85-c077-4d9e-9f59-0fa5b98d451d service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:14:36.352984 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29004d85-c077-4d9e-9f59-0fa5b98d451d service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:14:36.354080 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29004d85-c077-4d9e-9f59-0fa5b98d451d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:14:36.358081 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f53648-51b8-4281-9a20-d99468b37bb8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51733781201243e8abf9eca39b207750', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:14:36.359005 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f53648-51b8-4281-9a20-d99468b37bb8 service neutron] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:14:36.359904 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f53648-51b8-4281-9a20-d99468b37bb8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:14:36.360423 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f53648-51b8-4281-9a20-d99468b37bb8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:14:36.441643 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29004d85-c077-4d9e-9f59-0fa5b98d451d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:14:36.443427 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29004d85-c077-4d9e-9f59-0fa5b98d451d service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:14:36.447562 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:14:36.450019 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f53648-51b8-4281-9a20-d99468b37bb8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:14:36.451736 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f53648-51b8-4281-9a20-d99468b37bb8 service neutron] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:14:36.455830 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 809/1620] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:14:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:14:36.456924 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29004d85-c077-4d9e-9f59-0fa5b98d451d service neutron] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:14:36.466990 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29004d85-c077-4d9e-9f59-0fa5b98d451d service neutron] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:14:36.473420 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c259825d-9229-4215-9f57-1f19890ac4a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51733781201243e8abf9eca39b207750', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:14:36.473633 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 812/1621] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:14:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:14:36.473762 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c259825d-9229-4215-9f57-1f19890ac4a1 service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:14:36.473762 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c259825d-9229-4215-9f57-1f19890ac4a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:14:36.474061 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c259825d-9229-4215-9f57-1f19890ac4a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:14:36.475016 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b9b6a4-36dc-4507-bccb-5a96cb3ce042 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:14:36.475016 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b9b6a4-36dc-4507-bccb-5a96cb3ce042 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:14:36.475229 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b9b6a4-36dc-4507-bccb-5a96cb3ce042 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:14:36.477152 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 813/1622] 10.222.0.42 () {54 vars in 962 bytes} [Mon Jul 25 12:14:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:14:36.478120 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b9b6a4-36dc-4507-bccb-5a96cb3ce042 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:14:36.492806 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c259825d-9229-4215-9f57-1f19890ac4a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:14:36.495080 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b9b6a4-36dc-4507-bccb-5a96cb3ce042 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:14:36.495497 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c259825d-9229-4215-9f57-1f19890ac4a1 service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:14:36.502572 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f1a7c9-82c6-4b5a-b3f9-971eae6022e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:14:36.503594 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f1a7c9-82c6-4b5a-b3f9-971eae6022e0 admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:14:36.503978 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f1a7c9-82c6-4b5a-b3f9-971eae6022e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:14:36.504272 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f1a7c9-82c6-4b5a-b3f9-971eae6022e0 admin admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:14:36.505354 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f1a7c9-82c6-4b5a-b3f9-971eae6022e0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:14:36.508600 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f1a7c9-82c6-4b5a-b3f9-971eae6022e0 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:14:36.509261 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 810/1623] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:14:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:14:36.510427 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c259825d-9229-4215-9f57-1f19890ac4a1 service neutron] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:14:36.549622 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 814/1624] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Jul 25 12:14:36 2022] POST /identity/v3/users => generated 289 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:14:36.552531 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5aa6b9-a0f1-42f8-a5cb-47938cf0e9d3 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:14:36.552819 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5aa6b9-a0f1-42f8-a5cb-47938cf0e9d3 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:14:36.553149 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5aa6b9-a0f1-42f8-a5cb-47938cf0e9d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:14:36.584357 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c259825d-9229-4215-9f57-1f19890ac4a1 service neutron] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:14:36.589582 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812274de-63c5-4cff-a3e8-7b57bdb24530 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f89ce6f0cf04373a414d927fa0f8a7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:14:36.590346 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812274de-63c5-4cff-a3e8-7b57bdb24530 service cinder] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:14:36.590755 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812274de-63c5-4cff-a3e8-7b57bdb24530 service cinder] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:14:36.591276 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812274de-63c5-4cff-a3e8-7b57bdb24530 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:14:36.600966 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-0a5aa6b9-a0f1-42f8-a5cb-47938cf0e9d3 None None] MFA Rules not processed for user `f67142a1193d47c7bd33d7040268233c`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:14:36.604451 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-0a5aa6b9-a0f1-42f8-a5cb-47938cf0e9d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:14:36.629999 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:14:36.694445 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812274de-63c5-4cff-a3e8-7b57bdb24530 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:14:36.695908 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812274de-63c5-4cff-a3e8-7b57bdb24530 service cinder] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:14:36.710449 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 811/1625] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:14:36 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:14:36.711507 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812274de-63c5-4cff-a3e8-7b57bdb24530 service cinder] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:14:36.724411 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812274de-63c5-4cff-a3e8-7b57bdb24530 service cinder] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:14:36.730613 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25925356-e815-4dbe-9ab8-125757c52177 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:14:36.731439 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25925356-e815-4dbe-9ab8-125757c52177 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:14:36.732026 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25925356-e815-4dbe-9ab8-125757c52177 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:14:36.732446 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25925356-e815-4dbe-9ab8-125757c52177 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:14:36.733623 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25925356-e815-4dbe-9ab8-125757c52177 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:14:36.735420 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25925356-e815-4dbe-9ab8-125757c52177 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:14:36.749266 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 812/1626] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Jul 25 12:14:36 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:14:36.753824 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25925356-e815-4dbe-9ab8-125757c52177 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:14:36.776326 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25925356-e815-4dbe-9ab8-125757c52177 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:14:36.792978 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b63b7d6-832a-44b3-9f67-f4d0bfae95da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:14:36.795102 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b63b7d6-832a-44b3-9f67-f4d0bfae95da admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:14:36.796073 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b63b7d6-832a-44b3-9f67-f4d0bfae95da admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:14:36.798362 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b63b7d6-832a-44b3-9f67-f4d0bfae95da admin admin] PATH_INFO: `/v3/projects/bfc5031ea45a4e788cdaca9009f890c1/users/3d3ce0880f6541e9973e1b7228b43304/roles/a7d20885b6d94928a34b3cc5b3a00006` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:14:36.811262 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 815/1627] 10.222.0.42 () {58 vars in 1059 bytes} [Mon Jul 25 12:14:36 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:14:36.819774 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5aa6b9-a0f1-42f8-a5cb-47938cf0e9d3 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:14:36.832033 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b63b7d6-832a-44b3-9f67-f4d0bfae95da admin admin] RBAC: Authorizing `identity:create_grant(project_id=bfc5031ea45a4e788cdaca9009f890c1, user_id=3d3ce0880f6541e9973e1b7228b43304, role_id=a7d20885b6d94928a34b3cc5b3a00006)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:14:36.835511 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b63b7d6-832a-44b3-9f67-f4d0bfae95da admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:14:36.843288 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5aa6b9-a0f1-42f8-a5cb-47938cf0e9d3 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:14:36.861936 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5446a067-419e-458f-955f-166bb37786da service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f67142a1193d47c7bd33d7040268233c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:14:36.863428 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 813/1628] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Jul 25 12:14:36 2022] PUT /identity/v3/projects/bfc5031ea45a4e788cdaca9009f890c1/users/3d3ce0880f6541e9973e1b7228b43304/roles/a7d20885b6d94928a34b3cc5b3a00006 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:14:36.867341 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b63b7d6-832a-44b3-9f67-f4d0bfae95da admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:14:36.868423 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5446a067-419e-458f-955f-166bb37786da service nova] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:14:36.868817 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5446a067-419e-458f-955f-166bb37786da service nova] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:14:36.869126 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5446a067-419e-458f-955f-166bb37786da service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:14:36.961807 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b63b7d6-832a-44b3-9f67-f4d0bfae95da admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:14:36.969389 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f77aa5c-4cce-482c-9eba-d6a3f9f671cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:14:36.971095 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f77aa5c-4cce-482c-9eba-d6a3f9f671cb admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:14:36.971666 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f77aa5c-4cce-482c-9eba-d6a3f9f671cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:14:36.972498 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f77aa5c-4cce-482c-9eba-d6a3f9f671cb admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:14:36.973758 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f77aa5c-4cce-482c-9eba-d6a3f9f671cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:14:36.976024 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f77aa5c-4cce-482c-9eba-d6a3f9f671cb admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:14:37.003991 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 814/1629] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Jul 25 12:14:36 2022] GET /identity/v3/roles => generated 1420 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 12:14:37.008373 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f77aa5c-4cce-482c-9eba-d6a3f9f671cb admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:14:37.029356 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f77aa5c-4cce-482c-9eba-d6a3f9f671cb admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:14:37.038084 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5446a067-419e-458f-955f-166bb37786da service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:14:37.042770 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5446a067-419e-458f-955f-166bb37786da service nova] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:14:37.043530 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2918cc11-6552-4493-98f8-648334093c2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:14:37.047798 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2918cc11-6552-4493-98f8-648334093c2e admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:14:37.048178 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2918cc11-6552-4493-98f8-648334093c2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:14:37.048597 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2918cc11-6552-4493-98f8-648334093c2e admin admin] PATH_INFO: `/v3/projects/bfc5031ea45a4e788cdaca9009f890c1/users/3d3ce0880f6541e9973e1b7228b43304/roles/a7d20885b6d94928a34b3cc5b3a00006` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:14:37.053865 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2918cc11-6552-4493-98f8-648334093c2e admin admin] RBAC: Authorizing `identity:create_grant(project_id=bfc5031ea45a4e788cdaca9009f890c1, user_id=3d3ce0880f6541e9973e1b7228b43304, role_id=a7d20885b6d94928a34b3cc5b3a00006)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:14:37.062593 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2918cc11-6552-4493-98f8-648334093c2e admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:14:37.069133 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 816/1630] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:14:36 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 250 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:14:37.122984 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 815/1631] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Jul 25 12:14:37 2022] PUT /identity/v3/projects/bfc5031ea45a4e788cdaca9009f890c1/users/3d3ce0880f6541e9973e1b7228b43304/roles/a7d20885b6d94928a34b3cc5b3a00006 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:14:37.126155 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77c7214-ccaf-4fc8-9afb-9b42e59fd66b None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:14:37.126634 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77c7214-ccaf-4fc8-9afb-9b42e59fd66b None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:14:37.127081 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77c7214-ccaf-4fc8-9afb-9b42e59fd66b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:14:37.186274 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-c77c7214-ccaf-4fc8-9afb-9b42e59fd66b None None] MFA Rules not processed for user `3d3ce0880f6541e9973e1b7228b43304`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:14:37.188844 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-c77c7214-ccaf-4fc8-9afb-9b42e59fd66b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:14:37.308165 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:14:37.327437 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 817/1632] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:14:37 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:14:37.343163 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2918cc11-6552-4493-98f8-648334093c2e admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:14:37.519520 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2918cc11-6552-4493-98f8-648334093c2e admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:14:37.519520 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa54738-6d92-46ec-9758-31386dd8ef87 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f89ce6f0cf04373a414d927fa0f8a7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:14:37.519520 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa54738-6d92-46ec-9758-31386dd8ef87 service cinder] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:14:37.519520 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa54738-6d92-46ec-9758-31386dd8ef87 service cinder] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:14:37.532239 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa54738-6d92-46ec-9758-31386dd8ef87 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:14:37.532659 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a76221-eb1f-4694-885b-b0df9268c9d9 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:14:37.532659 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a76221-eb1f-4694-885b-b0df9268c9d9 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:14:37.532659 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a76221-eb1f-4694-885b-b0df9268c9d9 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:14:37.532659 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 818/1633] 10.222.0.42 () {54 vars in 912 bytes} [Mon Jul 25 12:14:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 12:14:37.546998 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d805df-e72f-485e-b87d-32ccf46e5e3c None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:14:37.547300 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d805df-e72f-485e-b87d-32ccf46e5e3c None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:14:37.547635 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d805df-e72f-485e-b87d-32ccf46e5e3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:14:37.562838 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa54738-6d92-46ec-9758-31386dd8ef87 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:14:37.571423 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa54738-6d92-46ec-9758-31386dd8ef87 service cinder] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:14:37.591717 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-75d805df-e72f-485e-b87d-32ccf46e5e3c None None] MFA Rules not processed for user `f67142a1193d47c7bd33d7040268233c`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:14:37.593269 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 816/1634] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:14:37 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 250 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:14:37.593976 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-75d805df-e72f-485e-b87d-32ccf46e5e3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:14:37.783960 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 819/1635] 10.222.0.42 () {58 vars in 1009 bytes} [Mon Jul 25 12:14:37 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 12:14:37.815118 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa54738-6d92-46ec-9758-31386dd8ef87 service cinder] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:14:37.828259 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa54738-6d92-46ec-9758-31386dd8ef87 service cinder] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:14:37.839201 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef95235f-971f-4535-83e0-11d98f545081 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f89ce6f0cf04373a414d927fa0f8a7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:14:37.839905 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef95235f-971f-4535-83e0-11d98f545081 service cinder] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:14:37.840250 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef95235f-971f-4535-83e0-11d98f545081 service cinder] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:14:37.840474 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef95235f-971f-4535-83e0-11d98f545081 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:14:37.871052 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:14:37.874012 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef95235f-971f-4535-83e0-11d98f545081 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:14:37.875751 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef95235f-971f-4535-83e0-11d98f545081 service cinder] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:14:37.906304 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 817/1636] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:14:37 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:14:38.668599 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d805df-e72f-485e-b87d-32ccf46e5e3c None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:14:38.814683 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d805df-e72f-485e-b87d-32ccf46e5e3c None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:14:38.821518 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba0e494-867e-4af1-b69d-c5cd0f89afeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:14:38.823137 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba0e494-867e-4af1-b69d-c5cd0f89afeb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:14:38.823613 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba0e494-867e-4af1-b69d-c5cd0f89afeb admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:14:38.824054 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba0e494-867e-4af1-b69d-c5cd0f89afeb admin admin] PATH_INFO: `/v3/users/880ee36229b1452f84928299f8b8926e` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:14:38.825790 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba0e494-867e-4af1-b69d-c5cd0f89afeb admin admin] RBAC: Authorizing `identity:delete_user(user_id=880ee36229b1452f84928299f8b8926e)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:14:38.827009 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: WARNING py.warnings [None req-dba0e494-867e-4af1-b69d-c5cd0f89afeb admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 25 12:14:38.827009 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: warnings.warn(msg) Jul 25 12:14:38.827009 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: Jul 25 12:14:38.828759 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba0e494-867e-4af1-b69d-c5cd0f89afeb admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:14:38.939391 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-dba0e494-867e-4af1-b69d-c5cd0f89afeb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '880ee36229b1452f84928299f8b8926e'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:14:38.940094 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-dba0e494-867e-4af1-b69d-c5cd0f89afeb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '880ee36229b1452f84928299f8b8926e'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:14:38.963049 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-dba0e494-867e-4af1-b69d-c5cd0f89afeb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '880ee36229b1452f84928299f8b8926e'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:14:39.068579 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-dba0e494-867e-4af1-b69d-c5cd0f89afeb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '880ee36229b1452f84928299f8b8926e'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:14:39.120959 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-dba0e494-867e-4af1-b69d-c5cd0f89afeb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '880ee36229b1452f84928299f8b8926e'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:14:39.139668 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 820/1637] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Jul 25 12:14:38 2022] DELETE /identity/v3/users/880ee36229b1452f84928299f8b8926e => generated 0 bytes in 471 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:14:39.144336 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37476bee-51e8-4c88-9a13-6f971a6438e8 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:14:39.144336 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37476bee-51e8-4c88-9a13-6f971a6438e8 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:14:39.144977 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37476bee-51e8-4c88-9a13-6f971a6438e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:14:39.196007 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-37476bee-51e8-4c88-9a13-6f971a6438e8 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:14:39.197864 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-37476bee-51e8-4c88-9a13-6f971a6438e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:14:39.358002 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 818/1638] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:14:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:14:39.424506 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba0e494-867e-4af1-b69d-c5cd0f89afeb admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:14:39.425838 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-dba0e494-867e-4af1-b69d-c5cd0f89afeb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:14:39.436683 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:14:39.545310 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba0e494-867e-4af1-b69d-c5cd0f89afeb admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:14:39.550721 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63356e04-3b0f-431a-b1a6-5e5481443706 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51733781201243e8abf9eca39b207750', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:14:39.551935 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63356e04-3b0f-431a-b1a6-5e5481443706 service neutron] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:14:39.552522 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63356e04-3b0f-431a-b1a6-5e5481443706 service neutron] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:14:39.552998 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63356e04-3b0f-431a-b1a6-5e5481443706 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:14:39.570267 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63356e04-3b0f-431a-b1a6-5e5481443706 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:14:39.577837 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63356e04-3b0f-431a-b1a6-5e5481443706 service neutron] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:14:39.591687 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 821/1639] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:14:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:14:41.994927 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37476bee-51e8-4c88-9a13-6f971a6438e8 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:14:41.996474 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-37476bee-51e8-4c88-9a13-6f971a6438e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:14:42.023764 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37476bee-51e8-4c88-9a13-6f971a6438e8 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:14:42.037199 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf8cd65-3ae2-4295-9c1d-be4e75e993b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:14:42.038650 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf8cd65-3ae2-4295-9c1d-be4e75e993b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:14:42.039028 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf8cd65-3ae2-4295-9c1d-be4e75e993b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:14:42.039458 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf8cd65-3ae2-4295-9c1d-be4e75e993b1 admin admin] PATH_INFO: `/v3/projects/13ebc33acf09437699e79e3ba39e14bb` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:14:42.042969 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf8cd65-3ae2-4295-9c1d-be4e75e993b1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=13ebc33acf09437699e79e3ba39e14bb)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:14:42.047017 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: WARNING py.warnings [None req-caf8cd65-3ae2-4295-9c1d-be4e75e993b1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 25 12:14:42.047017 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: warnings.warn(msg) Jul 25 12:14:42.047017 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: Jul 25 12:14:42.048314 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf8cd65-3ae2-4295-9c1d-be4e75e993b1 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:14:42.180741 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-caf8cd65-3ae2-4295-9c1d-be4e75e993b1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '13ebc33acf09437699e79e3ba39e14bb', 'request_id': 'req-caf8cd65-3ae2-4295-9c1d-be4e75e993b1'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:14:42.205169 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-caf8cd65-3ae2-4295-9c1d-be4e75e993b1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '13ebc33acf09437699e79e3ba39e14bb', 'request_id': 'req-caf8cd65-3ae2-4295-9c1d-be4e75e993b1'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:14:42.216347 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-caf8cd65-3ae2-4295-9c1d-be4e75e993b1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '13ebc33acf09437699e79e3ba39e14bb', 'request_id': 'req-caf8cd65-3ae2-4295-9c1d-be4e75e993b1'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:14:42.258245 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-caf8cd65-3ae2-4295-9c1d-be4e75e993b1 admin admin] The token cache is being invalidate because project 13ebc33acf09437699e79e3ba39e14bb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 12:14:42.258553 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-caf8cd65-3ae2-4295-9c1d-be4e75e993b1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:14:42.259330 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-caf8cd65-3ae2-4295-9c1d-be4e75e993b1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:14:42.261612 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 819/1640] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Jul 25 12:14:41 2022] DELETE /identity/v3/projects/13ebc33acf09437699e79e3ba39e14bb => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:14:42.266522 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63356e04-3b0f-431a-b1a6-5e5481443706 service neutron] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:14:42.284674 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63356e04-3b0f-431a-b1a6-5e5481443706 service neutron] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:14:42.428367 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3898fd74-321c-41d9-bc70-840f44ab6d96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:14:42.429279 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3898fd74-321c-41d9-bc70-840f44ab6d96 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:14:42.429565 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3898fd74-321c-41d9-bc70-840f44ab6d96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:14:42.429844 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3898fd74-321c-41d9-bc70-840f44ab6d96 admin admin] PATH_INFO: `/v3/users/3d3ce0880f6541e9973e1b7228b43304` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:14:42.432547 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3898fd74-321c-41d9-bc70-840f44ab6d96 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3d3ce0880f6541e9973e1b7228b43304)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:14:42.440144 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3898fd74-321c-41d9-bc70-840f44ab6d96 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:14:42.560953 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-3898fd74-321c-41d9-bc70-840f44ab6d96 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3d3ce0880f6541e9973e1b7228b43304'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:14:42.563307 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-3898fd74-321c-41d9-bc70-840f44ab6d96 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3d3ce0880f6541e9973e1b7228b43304'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:14:42.586173 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-3898fd74-321c-41d9-bc70-840f44ab6d96 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3d3ce0880f6541e9973e1b7228b43304'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:14:42.615272 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-3898fd74-321c-41d9-bc70-840f44ab6d96 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3d3ce0880f6541e9973e1b7228b43304'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:14:42.659529 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-3898fd74-321c-41d9-bc70-840f44ab6d96 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3d3ce0880f6541e9973e1b7228b43304'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:14:42.678424 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 822/1641] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Jul 25 12:14:42 2022] DELETE /identity/v3/users/3d3ce0880f6541e9973e1b7228b43304 => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:14:43.882498 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf8cd65-3ae2-4295-9c1d-be4e75e993b1 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:14:43.884937 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-caf8cd65-3ae2-4295-9c1d-be4e75e993b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:14:43.910146 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf8cd65-3ae2-4295-9c1d-be4e75e993b1 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:14:44.011966 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b0e3d3-b2d1-42bf-a6d2-6d62bfdc77fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:14:44.013035 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b0e3d3-b2d1-42bf-a6d2-6d62bfdc77fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:14:44.013599 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b0e3d3-b2d1-42bf-a6d2-6d62bfdc77fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:14:44.014031 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b0e3d3-b2d1-42bf-a6d2-6d62bfdc77fa admin admin] PATH_INFO: `/v3/users/196c9e67f28046ee98a08a39d0f1e4ba` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:14:44.015524 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b0e3d3-b2d1-42bf-a6d2-6d62bfdc77fa admin admin] RBAC: Authorizing `identity:delete_user(user_id=196c9e67f28046ee98a08a39d0f1e4ba)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:14:44.017684 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b0e3d3-b2d1-42bf-a6d2-6d62bfdc77fa admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:14:44.047254 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:14:44.174113 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-05b0e3d3-b2d1-42bf-a6d2-6d62bfdc77fa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '196c9e67f28046ee98a08a39d0f1e4ba'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:14:44.189886 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-05b0e3d3-b2d1-42bf-a6d2-6d62bfdc77fa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '196c9e67f28046ee98a08a39d0f1e4ba'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:14:44.211494 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-05b0e3d3-b2d1-42bf-a6d2-6d62bfdc77fa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '196c9e67f28046ee98a08a39d0f1e4ba'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:14:44.240030 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-05b0e3d3-b2d1-42bf-a6d2-6d62bfdc77fa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '196c9e67f28046ee98a08a39d0f1e4ba'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:14:44.242119 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-05b0e3d3-b2d1-42bf-a6d2-6d62bfdc77fa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '196c9e67f28046ee98a08a39d0f1e4ba'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:14:44.264555 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 820/1642] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Jul 25 12:14:43 2022] DELETE /identity/v3/users/196c9e67f28046ee98a08a39d0f1e4ba => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:14:44.269516 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61f5ee9-a9d3-4a21-ba23-5360fa3956ba None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:14:44.272930 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61f5ee9-a9d3-4a21-ba23-5360fa3956ba None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:14:44.273443 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61f5ee9-a9d3-4a21-ba23-5360fa3956ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:14:44.343843 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-a61f5ee9-a9d3-4a21-ba23-5360fa3956ba None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:14:44.345754 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-a61f5ee9-a9d3-4a21-ba23-5360fa3956ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:14:44.475638 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:14:44.630048 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 823/1643] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:14:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 364 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:14:44.685901 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b0e3d3-b2d1-42bf-a6d2-6d62bfdc77fa admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:14:44.687854 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-05b0e3d3-b2d1-42bf-a6d2-6d62bfdc77fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:14:44.858748 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61f5ee9-a9d3-4a21-ba23-5360fa3956ba None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:14:44.864427 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-a61f5ee9-a9d3-4a21-ba23-5360fa3956ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:14:44.889146 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b0e3d3-b2d1-42bf-a6d2-6d62bfdc77fa admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:14:44.904979 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61f5ee9-a9d3-4a21-ba23-5360fa3956ba None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:14:44.910391 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb45b02-01ef-4425-8577-607999671a8c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51733781201243e8abf9eca39b207750', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:14:44.911442 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb45b02-01ef-4425-8577-607999671a8c service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:14:44.911990 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb45b02-01ef-4425-8577-607999671a8c service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:14:44.912785 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb45b02-01ef-4425-8577-607999671a8c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:14:44.921466 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c38a046-5338-4ce7-bc07-cb3d40878a32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:14:44.922742 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c38a046-5338-4ce7-bc07-cb3d40878a32 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:14:44.924307 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c38a046-5338-4ce7-bc07-cb3d40878a32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:14:44.926396 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c38a046-5338-4ce7-bc07-cb3d40878a32 admin admin] PATH_INFO: `/v3/projects/bfc5031ea45a4e788cdaca9009f890c1` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:14:44.941610 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb45b02-01ef-4425-8577-607999671a8c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:14:44.943495 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c38a046-5338-4ce7-bc07-cb3d40878a32 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bfc5031ea45a4e788cdaca9009f890c1)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:14:44.944182 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb45b02-01ef-4425-8577-607999671a8c service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:14:44.946201 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c38a046-5338-4ce7-bc07-cb3d40878a32 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:14:44.971750 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 821/1644] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:14:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 287 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:14:45.050927 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a5a692-35e5-4cb9-bac7-1fedf68d0549 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:14:45.052053 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a5a692-35e5-4cb9-bac7-1fedf68d0549 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:14:45.052540 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a5a692-35e5-4cb9-bac7-1fedf68d0549 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:14:45.078799 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-4c38a046-5338-4ce7-bc07-cb3d40878a32 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bfc5031ea45a4e788cdaca9009f890c1', 'request_id': 'req-4c38a046-5338-4ce7-bc07-cb3d40878a32'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:14:45.100057 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-4c38a046-5338-4ce7-bc07-cb3d40878a32 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bfc5031ea45a4e788cdaca9009f890c1', 'request_id': 'req-4c38a046-5338-4ce7-bc07-cb3d40878a32'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:14:45.108891 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-4c38a046-5338-4ce7-bc07-cb3d40878a32 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bfc5031ea45a4e788cdaca9009f890c1', 'request_id': 'req-4c38a046-5338-4ce7-bc07-cb3d40878a32'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:14:45.122886 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-07a5a692-35e5-4cb9-bac7-1fedf68d0549 None None] MFA Rules not processed for user `d2d6f10a6fe841e5b1d12bce78f97d91`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:14:45.124296 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-07a5a692-35e5-4cb9-bac7-1fedf68d0549 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:14:45.131827 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-4c38a046-5338-4ce7-bc07-cb3d40878a32 admin admin] The token cache is being invalidate because project bfc5031ea45a4e788cdaca9009f890c1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80298) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 12:14:45.132132 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-4c38a046-5338-4ce7-bc07-cb3d40878a32 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:14:45.134169 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-4c38a046-5338-4ce7-bc07-cb3d40878a32 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:14:45.139969 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 824/1645] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Jul 25 12:14:44 2022] DELETE /identity/v3/projects/bfc5031ea45a4e788cdaca9009f890c1 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:14:45.145897 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c380d5c-7310-427d-bd56-dca54186b41d None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:14:45.146194 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c380d5c-7310-427d-bd56-dca54186b41d None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:14:45.146482 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c380d5c-7310-427d-bd56-dca54186b41d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:14:45.212543 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-8c380d5c-7310-427d-bd56-dca54186b41d None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:14:45.213808 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-8c380d5c-7310-427d-bd56-dca54186b41d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:14:45.279436 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 822/1646] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:14:45 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:14:45.297380 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a5a692-35e5-4cb9-bac7-1fedf68d0549 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:14:45.298638 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-07a5a692-35e5-4cb9-bac7-1fedf68d0549 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:14:45.380755 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:14:45.394921 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 825/1647] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:14:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:14:45.404215 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c86ca47-82ed-48c8-b578-b57f85651a68 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:14:45.404557 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c86ca47-82ed-48c8-b578-b57f85651a68 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:14:45.404790 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c86ca47-82ed-48c8-b578-b57f85651a68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:14:45.471255 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-6c86ca47-82ed-48c8-b578-b57f85651a68 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:14:45.473501 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-6c86ca47-82ed-48c8-b578-b57f85651a68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:14:45.485421 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a5a692-35e5-4cb9-bac7-1fedf68d0549 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:14:45.503911 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 826/1648] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:14:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 12:14:45.507977 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c86ca47-82ed-48c8-b578-b57f85651a68 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:14:45.509876 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949b8449-0328-4c12-abb4-68407f00e82f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f67142a1193d47c7bd33d7040268233c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:14:45.515792 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949b8449-0328-4c12-abb4-68407f00e82f service nova] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:14:45.516324 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949b8449-0328-4c12-abb4-68407f00e82f service nova] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:14:45.516765 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949b8449-0328-4c12-abb4-68407f00e82f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:14:45.525815 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c86ca47-82ed-48c8-b578-b57f85651a68 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:14:45.537505 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949b8449-0328-4c12-abb4-68407f00e82f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:14:45.538929 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949b8449-0328-4c12-abb4-68407f00e82f service nova] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:14:45.549859 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7430bd-add6-438d-b0da-b87e7ae5e1a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:14:45.553318 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7430bd-add6-438d-b0da-b87e7ae5e1a8 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:14:45.554209 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7430bd-add6-438d-b0da-b87e7ae5e1a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:14:45.554721 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7430bd-add6-438d-b0da-b87e7ae5e1a8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:14:45.556900 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7430bd-add6-438d-b0da-b87e7ae5e1a8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:14:45.557083 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7430bd-add6-438d-b0da-b87e7ae5e1a8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:14:45.566900 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7430bd-add6-438d-b0da-b87e7ae5e1a8 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:14:45.574248 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 823/1649] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:14:45 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 277 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:14:45.619063 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 827/1650] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Jul 25 12:14:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 12:14:45.622623 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949b8449-0328-4c12-abb4-68407f00e82f service nova] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:14:45.643890 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949b8449-0328-4c12-abb4-68407f00e82f service nova] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:14:45.672288 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e73b62-daae-40f4-b985-cfe5f049e4b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:14:45.672994 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e73b62-daae-40f4-b985-cfe5f049e4b1 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:14:45.673286 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e73b62-daae-40f4-b985-cfe5f049e4b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:14:45.673565 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e73b62-daae-40f4-b985-cfe5f049e4b1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:14:45.674275 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e73b62-daae-40f4-b985-cfe5f049e4b1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:14:45.684588 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e73b62-daae-40f4-b985-cfe5f049e4b1 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:14:45.748558 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 824/1651] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Jul 25 12:14:45 2022] POST /identity/v3/projects => generated 365 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:14:45.749601 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7430bd-add6-438d-b0da-b87e7ae5e1a8 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:14:45.771930 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7430bd-add6-438d-b0da-b87e7ae5e1a8 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:14:45.866348 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:14:46.027224 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a10b02-a400-45b0-85ea-0060ef6b349f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:14:46.027974 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a10b02-a400-45b0-85ea-0060ef6b349f admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:14:46.028428 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a10b02-a400-45b0-85ea-0060ef6b349f admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:14:46.028659 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a10b02-a400-45b0-85ea-0060ef6b349f admin admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:14:46.029323 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a10b02-a400-45b0-85ea-0060ef6b349f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:14:46.031230 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a10b02-a400-45b0-85ea-0060ef6b349f admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:14:46.140562 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 828/1652] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Jul 25 12:14:45 2022] POST /identity/v3/users => generated 283 bytes in 390 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 12:14:46.143448 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e73b62-daae-40f4-b985-cfe5f049e4b1 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:14:46.143970 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a10b02-a400-45b0-85ea-0060ef6b349f admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:14:46.145942 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-84e73b62-daae-40f4-b985-cfe5f049e4b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:14:46.163286 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a10b02-a400-45b0-85ea-0060ef6b349f admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:14:46.179393 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e73b62-daae-40f4-b985-cfe5f049e4b1 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:14:46.182227 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8bc13c-63bc-4909-baa6-5fe56143ba25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:14:46.182508 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8bc13c-63bc-4909-baa6-5fe56143ba25 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:14:46.183074 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8bc13c-63bc-4909-baa6-5fe56143ba25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:14:46.183365 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8bc13c-63bc-4909-baa6-5fe56143ba25 admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:14:46.184146 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8bc13c-63bc-4909-baa6-5fe56143ba25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:14:46.185797 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8bc13c-63bc-4909-baa6-5fe56143ba25 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:14:46.201632 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80395ecc-c205-4730-b76f-fba25ab6ff94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:14:46.202816 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80395ecc-c205-4730-b76f-fba25ab6ff94 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:14:46.203215 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80395ecc-c205-4730-b76f-fba25ab6ff94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:14:46.203736 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80395ecc-c205-4730-b76f-fba25ab6ff94 admin admin] PATH_INFO: `/v3/users/b7c740c7ce2e4b3c984247b6b2b69da1` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:14:46.206776 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80395ecc-c205-4730-b76f-fba25ab6ff94 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b7c740c7ce2e4b3c984247b6b2b69da1)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:14:46.213488 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80395ecc-c205-4730-b76f-fba25ab6ff94 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:14:46.228678 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 829/1653] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Jul 25 12:14:46 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 12:14:46.232101 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8bc13c-63bc-4909-baa6-5fe56143ba25 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:14:46.252851 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8bc13c-63bc-4909-baa6-5fe56143ba25 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:14:46.266391 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a885307-a1ec-4499-88c5-e03387b15268 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:14:46.267110 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a885307-a1ec-4499-88c5-e03387b15268 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:14:46.267417 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a885307-a1ec-4499-88c5-e03387b15268 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:14:46.267717 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a885307-a1ec-4499-88c5-e03387b15268 admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:14:46.268461 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a885307-a1ec-4499-88c5-e03387b15268 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:14:46.273249 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a885307-a1ec-4499-88c5-e03387b15268 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:14:46.301972 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 830/1654] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Jul 25 12:14:46 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:14:46.306932 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a885307-a1ec-4499-88c5-e03387b15268 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:14:46.330088 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a885307-a1ec-4499-88c5-e03387b15268 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:14:46.343206 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68684ec8-1b02-49b3-8af7-8433cce6203e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:14:46.344195 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68684ec8-1b02-49b3-8af7-8433cce6203e admin admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:14:46.344486 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68684ec8-1b02-49b3-8af7-8433cce6203e admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:14:46.344874 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68684ec8-1b02-49b3-8af7-8433cce6203e admin admin] PATH_INFO: `/v3/projects/6d8cb4a828cd49619c5f85a8e8881bc5/users/921e5815a8044de4a08927ddf18371dc/roles/472313ae69174578824df0bed6390744` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:14:46.380488 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68684ec8-1b02-49b3-8af7-8433cce6203e admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d8cb4a828cd49619c5f85a8e8881bc5, user_id=921e5815a8044de4a08927ddf18371dc, role_id=472313ae69174578824df0bed6390744)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:14:46.383423 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68684ec8-1b02-49b3-8af7-8433cce6203e admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:14:46.388903 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-80395ecc-c205-4730-b76f-fba25ab6ff94 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b7c740c7ce2e4b3c984247b6b2b69da1'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:14:46.398904 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-80395ecc-c205-4730-b76f-fba25ab6ff94 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b7c740c7ce2e4b3c984247b6b2b69da1'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:14:46.409216 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-80395ecc-c205-4730-b76f-fba25ab6ff94 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b7c740c7ce2e4b3c984247b6b2b69da1'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:14:46.421665 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 831/1655] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Jul 25 12:14:46 2022] PUT /identity/v3/projects/6d8cb4a828cd49619c5f85a8e8881bc5/users/921e5815a8044de4a08927ddf18371dc/roles/472313ae69174578824df0bed6390744 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:14:46.432380 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-80395ecc-c205-4730-b76f-fba25ab6ff94 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b7c740c7ce2e4b3c984247b6b2b69da1'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:14:46.432755 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f355313b-97b0-41cb-a060-dfa1efb971d1 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:14:46.433055 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f355313b-97b0-41cb-a060-dfa1efb971d1 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:14:46.433343 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f355313b-97b0-41cb-a060-dfa1efb971d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:14:46.433438 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-80395ecc-c205-4730-b76f-fba25ab6ff94 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b7c740c7ce2e4b3c984247b6b2b69da1'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:14:46.445638 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 825/1656] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Jul 25 12:14:46 2022] DELETE /identity/v3/users/b7c740c7ce2e4b3c984247b6b2b69da1 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:14:46.448744 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762840d3-2250-48ff-a85b-30b78380c09b None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:14:46.449027 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762840d3-2250-48ff-a85b-30b78380c09b None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:14:46.449399 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762840d3-2250-48ff-a85b-30b78380c09b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:14:46.496630 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-762840d3-2250-48ff-a85b-30b78380c09b None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:14:46.498157 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-762840d3-2250-48ff-a85b-30b78380c09b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:14:46.516221 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-f355313b-97b0-41cb-a060-dfa1efb971d1 None None] MFA Rules not processed for user `921e5815a8044de4a08927ddf18371dc`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:14:46.517401 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-f355313b-97b0-41cb-a060-dfa1efb971d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:14:46.636688 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 826/1657] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:14:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:14:46.655971 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762840d3-2250-48ff-a85b-30b78380c09b None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:14:46.657948 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-762840d3-2250-48ff-a85b-30b78380c09b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:14:46.677097 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:14:46.699060 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 832/1658] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:14:46 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 12:14:46.702403 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f355313b-97b0-41cb-a060-dfa1efb971d1 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:14:46.704387 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-f355313b-97b0-41cb-a060-dfa1efb971d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:14:46.742817 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f355313b-97b0-41cb-a060-dfa1efb971d1 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:14:46.758697 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73637738-9261-4bf6-bce5-f8467dc70799 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:14:46.764153 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73637738-9261-4bf6-bce5-f8467dc70799 admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:14:46.764524 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73637738-9261-4bf6-bce5-f8467dc70799 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:14:46.765353 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73637738-9261-4bf6-bce5-f8467dc70799 admin admin] PATH_INFO: `/v3/projects` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:14:46.767665 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73637738-9261-4bf6-bce5-f8467dc70799 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:14:46.776689 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73637738-9261-4bf6-bce5-f8467dc70799 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:14:46.819276 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762840d3-2250-48ff-a85b-30b78380c09b None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:14:46.845224 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1bf6e1-7aeb-43b9-ada2-1441d2a51b3d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51733781201243e8abf9eca39b207750', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:14:46.846751 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1bf6e1-7aeb-43b9-ada2-1441d2a51b3d service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:14:46.847055 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1bf6e1-7aeb-43b9-ada2-1441d2a51b3d service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:14:46.847340 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1bf6e1-7aeb-43b9-ada2-1441d2a51b3d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:14:46.860278 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 833/1659] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Jul 25 12:14:46 2022] POST /identity/v3/projects => generated 365 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:14:46.864671 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73637738-9261-4bf6-bce5-f8467dc70799 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:14:46.869518 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1bf6e1-7aeb-43b9-ada2-1441d2a51b3d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:14:46.871118 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1bf6e1-7aeb-43b9-ada2-1441d2a51b3d service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:14:46.887370 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 827/1660] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:14:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:14:46.949179 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:14:46.989464 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73637738-9261-4bf6-bce5-f8467dc70799 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:14:47.014152 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b2aa4c-7e5d-44f6-99c3-c7c930eee4ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:14:47.015196 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b2aa4c-7e5d-44f6-99c3-c7c930eee4ff admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:14:47.015718 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b2aa4c-7e5d-44f6-99c3-c7c930eee4ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:14:47.016219 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b2aa4c-7e5d-44f6-99c3-c7c930eee4ff admin admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:14:47.017104 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b2aa4c-7e5d-44f6-99c3-c7c930eee4ff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:14:47.019256 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b2aa4c-7e5d-44f6-99c3-c7c930eee4ff admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:14:47.065838 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 834/1661] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Jul 25 12:14:46 2022] POST /identity/v3/users => generated 289 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 12:14:47.070380 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1bf6e1-7aeb-43b9-ada2-1441d2a51b3d service neutron] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:14:47.105478 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1bf6e1-7aeb-43b9-ada2-1441d2a51b3d service neutron] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:14:47.132484 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3278f946-ad3d-44e2-8398-f78b893b4736 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:14:47.134621 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3278f946-ad3d-44e2-8398-f78b893b4736 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:14:47.134937 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3278f946-ad3d-44e2-8398-f78b893b4736 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:14:47.135253 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3278f946-ad3d-44e2-8398-f78b893b4736 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:14:47.139761 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3278f946-ad3d-44e2-8398-f78b893b4736 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:14:47.144770 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3278f946-ad3d-44e2-8398-f78b893b4736 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:14:47.185789 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 828/1662] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Jul 25 12:14:47 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:14:47.187293 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b2aa4c-7e5d-44f6-99c3-c7c930eee4ff admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:14:47.204492 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b2aa4c-7e5d-44f6-99c3-c7c930eee4ff admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:14:47.219450 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d742be-8b37-4424-bdf3-57a6d95f3c0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:14:47.220673 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d742be-8b37-4424-bdf3-57a6d95f3c0e admin admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:14:47.221143 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d742be-8b37-4424-bdf3-57a6d95f3c0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:14:47.221590 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d742be-8b37-4424-bdf3-57a6d95f3c0e admin admin] PATH_INFO: `/v3/projects/c2b8c9f9bfeb40baa4237d96504bd4dd/users/885dac3726694b288b0738d68d98223c/roles/a7d20885b6d94928a34b3cc5b3a00006` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:14:47.272290 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d742be-8b37-4424-bdf3-57a6d95f3c0e admin admin] RBAC: Authorizing `identity:create_grant(project_id=c2b8c9f9bfeb40baa4237d96504bd4dd, user_id=885dac3726694b288b0738d68d98223c, role_id=a7d20885b6d94928a34b3cc5b3a00006)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:14:47.276502 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d742be-8b37-4424-bdf3-57a6d95f3c0e admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:14:47.324219 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 835/1663] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Jul 25 12:14:47 2022] PUT /identity/v3/projects/c2b8c9f9bfeb40baa4237d96504bd4dd/users/885dac3726694b288b0738d68d98223c/roles/a7d20885b6d94928a34b3cc5b3a00006 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:14:47.324997 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3278f946-ad3d-44e2-8398-f78b893b4736 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:14:47.339872 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3278f946-ad3d-44e2-8398-f78b893b4736 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:14:47.449985 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b25aba-1ab5-4574-94df-46091c137434 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:14:47.451070 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b25aba-1ab5-4574-94df-46091c137434 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:14:47.451578 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b25aba-1ab5-4574-94df-46091c137434 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:14:47.452071 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b25aba-1ab5-4574-94df-46091c137434 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:14:47.453092 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b25aba-1ab5-4574-94df-46091c137434 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:14:47.455380 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b25aba-1ab5-4574-94df-46091c137434 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:14:47.474832 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d742be-8b37-4424-bdf3-57a6d95f3c0e admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:14:47.476949 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 829/1664] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Jul 25 12:14:47 2022] GET /identity/v3/roles => generated 1420 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 12:14:47.492123 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d742be-8b37-4424-bdf3-57a6d95f3c0e admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:14:47.504413 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3baef73f-c05f-4d54-a05a-c3e87118f6e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:14:47.505527 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3baef73f-c05f-4d54-a05a-c3e87118f6e1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:14:47.505926 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3baef73f-c05f-4d54-a05a-c3e87118f6e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:14:47.506312 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3baef73f-c05f-4d54-a05a-c3e87118f6e1 admin admin] PATH_INFO: `/v3/projects/c2b8c9f9bfeb40baa4237d96504bd4dd/users/885dac3726694b288b0738d68d98223c/roles/a7d20885b6d94928a34b3cc5b3a00006` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:14:47.512011 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3baef73f-c05f-4d54-a05a-c3e87118f6e1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c2b8c9f9bfeb40baa4237d96504bd4dd, user_id=885dac3726694b288b0738d68d98223c, role_id=a7d20885b6d94928a34b3cc5b3a00006)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:14:47.514650 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3baef73f-c05f-4d54-a05a-c3e87118f6e1 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:14:47.550470 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 836/1665] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Jul 25 12:14:47 2022] PUT /identity/v3/projects/c2b8c9f9bfeb40baa4237d96504bd4dd/users/885dac3726694b288b0738d68d98223c/roles/a7d20885b6d94928a34b3cc5b3a00006 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:14:47.552707 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71acdb3e-c2ca-4ec5-8282-4a640ec1db54 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:14:47.552707 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71acdb3e-c2ca-4ec5-8282-4a640ec1db54 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:14:47.552907 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71acdb3e-c2ca-4ec5-8282-4a640ec1db54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:14:47.610787 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-71acdb3e-c2ca-4ec5-8282-4a640ec1db54 None None] MFA Rules not processed for user `885dac3726694b288b0738d68d98223c`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:14:47.612570 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-71acdb3e-c2ca-4ec5-8282-4a640ec1db54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:14:47.726427 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 830/1666] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:14:47 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 12:14:47.735112 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3baef73f-c05f-4d54-a05a-c3e87118f6e1 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:14:47.737219 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-3baef73f-c05f-4d54-a05a-c3e87118f6e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:14:47.888774 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3baef73f-c05f-4d54-a05a-c3e87118f6e1 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:14:47.904870 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe73e62-c6aa-4551-9c25-31fc5f71a373 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f89ce6f0cf04373a414d927fa0f8a7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:14:47.905561 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe73e62-c6aa-4551-9c25-31fc5f71a373 service cinder] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:14:47.905881 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe73e62-c6aa-4551-9c25-31fc5f71a373 service cinder] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:14:47.906139 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe73e62-c6aa-4551-9c25-31fc5f71a373 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:14:47.932309 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe73e62-c6aa-4551-9c25-31fc5f71a373 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:14:47.933606 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe73e62-c6aa-4551-9c25-31fc5f71a373 service cinder] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:14:47.967208 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 837/1667] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:14:47 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:14:48.097768 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71acdb3e-c2ca-4ec5-8282-4a640ec1db54 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:14:48.113293 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71acdb3e-c2ca-4ec5-8282-4a640ec1db54 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:14:48.134648 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19d4d8d-f6b1-4924-a057-053dda10fbb2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f89ce6f0cf04373a414d927fa0f8a7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:14:48.135520 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19d4d8d-f6b1-4924-a057-053dda10fbb2 service cinder] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:14:48.135785 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19d4d8d-f6b1-4924-a057-053dda10fbb2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:14:48.136071 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19d4d8d-f6b1-4924-a057-053dda10fbb2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:14:48.180650 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:14:48.300153 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19d4d8d-f6b1-4924-a057-053dda10fbb2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:14:48.302447 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19d4d8d-f6b1-4924-a057-053dda10fbb2 service cinder] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:14:48.338968 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 831/1668] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:14:48 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 241 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:14:49.012299 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe73e62-c6aa-4551-9c25-31fc5f71a373 service cinder] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:14:49.014403 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-ebe73e62-c6aa-4551-9c25-31fc5f71a373 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:14:49.188396 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe73e62-c6aa-4551-9c25-31fc5f71a373 service cinder] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:14:49.204399 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30d6919-1a96-4b72-96b1-21d1e399cdf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:14:49.205316 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30d6919-1a96-4b72-96b1-21d1e399cdf4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:14:49.205609 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30d6919-1a96-4b72-96b1-21d1e399cdf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:14:49.205897 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30d6919-1a96-4b72-96b1-21d1e399cdf4 admin admin] PATH_INFO: `/v3/projects/e7e349c1736b41e5ac722809adbf3c69` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:14:49.207499 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30d6919-1a96-4b72-96b1-21d1e399cdf4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e7e349c1736b41e5ac722809adbf3c69)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:14:49.209622 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30d6919-1a96-4b72-96b1-21d1e399cdf4 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:14:49.297985 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:14:49.313705 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-a30d6919-1a96-4b72-96b1-21d1e399cdf4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e7e349c1736b41e5ac722809adbf3c69', 'request_id': 'req-a30d6919-1a96-4b72-96b1-21d1e399cdf4'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:14:49.334018 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-a30d6919-1a96-4b72-96b1-21d1e399cdf4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e7e349c1736b41e5ac722809adbf3c69', 'request_id': 'req-a30d6919-1a96-4b72-96b1-21d1e399cdf4'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:14:49.341928 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-a30d6919-1a96-4b72-96b1-21d1e399cdf4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e7e349c1736b41e5ac722809adbf3c69', 'request_id': 'req-a30d6919-1a96-4b72-96b1-21d1e399cdf4'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:14:49.365207 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19d4d8d-f6b1-4924-a057-053dda10fbb2 service cinder] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:14:49.366672 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-b19d4d8d-f6b1-4924-a057-053dda10fbb2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:14:49.381060 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-a30d6919-1a96-4b72-96b1-21d1e399cdf4 admin admin] The token cache is being invalidate because project e7e349c1736b41e5ac722809adbf3c69 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80298) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 12:14:49.381344 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-a30d6919-1a96-4b72-96b1-21d1e399cdf4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:14:49.388208 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-a30d6919-1a96-4b72-96b1-21d1e399cdf4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:14:49.393875 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 838/1669] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Jul 25 12:14:49 2022] DELETE /identity/v3/projects/e7e349c1736b41e5ac722809adbf3c69 => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:14:49.395285 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa13873-7c4c-4039-8804-a23df61d88f1 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:14:49.395605 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa13873-7c4c-4039-8804-a23df61d88f1 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:14:49.395896 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa13873-7c4c-4039-8804-a23df61d88f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:14:49.409400 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19d4d8d-f6b1-4924-a057-053dda10fbb2 service cinder] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:14:49.472165 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-efa13873-7c4c-4039-8804-a23df61d88f1 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:14:49.473434 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-efa13873-7c4c-4039-8804-a23df61d88f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:14:49.661212 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 839/1670] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:14:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 267 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:14:49.662702 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc5b4ed-bff4-41da-9111-43fb9e04c163 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:14:49.663003 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc5b4ed-bff4-41da-9111-43fb9e04c163 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:14:49.663282 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc5b4ed-bff4-41da-9111-43fb9e04c163 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:14:49.689386 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207c2aa9-172e-44ba-bc68-6a10059b8c73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:14:49.690281 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207c2aa9-172e-44ba-bc68-6a10059b8c73 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:14:49.690755 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207c2aa9-172e-44ba-bc68-6a10059b8c73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:14:49.691179 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207c2aa9-172e-44ba-bc68-6a10059b8c73 admin admin] PATH_INFO: `/v3/projects/6ceed8120f864c4aa210ae72a2b1e9c5` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:14:49.693484 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207c2aa9-172e-44ba-bc68-6a10059b8c73 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6ceed8120f864c4aa210ae72a2b1e9c5)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:14:49.695596 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207c2aa9-172e-44ba-bc68-6a10059b8c73 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:14:49.724100 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-3cc5b4ed-bff4-41da-9111-43fb9e04c163 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:14:49.725666 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-3cc5b4ed-bff4-41da-9111-43fb9e04c163 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:14:49.755626 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 840/1671] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:14:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:14:49.760078 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc5b4ed-bff4-41da-9111-43fb9e04c163 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:14:49.779667 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc5b4ed-bff4-41da-9111-43fb9e04c163 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:14:49.791491 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fcc666-f37a-45eb-9d57-bfbf025b9c2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:14:49.792791 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fcc666-f37a-45eb-9d57-bfbf025b9c2c admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:14:49.793417 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fcc666-f37a-45eb-9d57-bfbf025b9c2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:14:49.794009 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fcc666-f37a-45eb-9d57-bfbf025b9c2c admin admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:14:49.795720 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fcc666-f37a-45eb-9d57-bfbf025b9c2c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:14:49.796416 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fcc666-f37a-45eb-9d57-bfbf025b9c2c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:14:49.798436 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fcc666-f37a-45eb-9d57-bfbf025b9c2c admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:14:49.822214 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 841/1672] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Jul 25 12:14:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 12:14:49.831436 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fcc666-f37a-45eb-9d57-bfbf025b9c2c admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:14:49.856608 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fcc666-f37a-45eb-9d57-bfbf025b9c2c admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:14:49.885214 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-207c2aa9-172e-44ba-bc68-6a10059b8c73 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6ceed8120f864c4aa210ae72a2b1e9c5', 'request_id': 'req-207c2aa9-172e-44ba-bc68-6a10059b8c73'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:14:49.912603 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-207c2aa9-172e-44ba-bc68-6a10059b8c73 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6ceed8120f864c4aa210ae72a2b1e9c5', 'request_id': 'req-207c2aa9-172e-44ba-bc68-6a10059b8c73'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:14:49.926679 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-207c2aa9-172e-44ba-bc68-6a10059b8c73 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6ceed8120f864c4aa210ae72a2b1e9c5', 'request_id': 'req-207c2aa9-172e-44ba-bc68-6a10059b8c73'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:14:49.964704 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-207c2aa9-172e-44ba-bc68-6a10059b8c73 admin admin] The token cache is being invalidate because project 6ceed8120f864c4aa210ae72a2b1e9c5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 12:14:49.965179 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-207c2aa9-172e-44ba-bc68-6a10059b8c73 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:14:49.966027 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-207c2aa9-172e-44ba-bc68-6a10059b8c73 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:14:49.971216 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 832/1673] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Jul 25 12:14:49 2022] DELETE /identity/v3/projects/6ceed8120f864c4aa210ae72a2b1e9c5 => generated 0 bytes in 604 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:14:49.972958 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d11f583-c0cf-4a11-ab38-f02fa448237f None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:14:49.973394 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d11f583-c0cf-4a11-ab38-f02fa448237f None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:14:49.973855 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d11f583-c0cf-4a11-ab38-f02fa448237f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:14:50.017244 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-2d11f583-c0cf-4a11-ab38-f02fa448237f None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:14:50.022376 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-2d11f583-c0cf-4a11-ab38-f02fa448237f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:14:50.025144 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbc5785-7ea1-4201-97bf-dbf3fadd92e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:14:50.031879 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbc5785-7ea1-4201-97bf-dbf3fadd92e1 admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:14:50.032309 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbc5785-7ea1-4201-97bf-dbf3fadd92e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:14:50.032708 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbc5785-7ea1-4201-97bf-dbf3fadd92e1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:14:50.033540 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbc5785-7ea1-4201-97bf-dbf3fadd92e1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:14:50.036837 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbc5785-7ea1-4201-97bf-dbf3fadd92e1 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:14:50.042600 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 833/1674] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:14:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:14:50.052888 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b531fb-85fb-468b-935e-cc1a6deff7f4 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:14:50.056253 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b531fb-85fb-468b-935e-cc1a6deff7f4 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:14:50.056724 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b531fb-85fb-468b-935e-cc1a6deff7f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:14:50.098366 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-b6b531fb-85fb-468b-935e-cc1a6deff7f4 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:14:50.101577 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-b6b531fb-85fb-468b-935e-cc1a6deff7f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:14:50.109932 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 842/1675] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Jul 25 12:14:49 2022] POST /identity/v3/projects => generated 367 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 12:14:50.111449 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbc5785-7ea1-4201-97bf-dbf3fadd92e1 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:14:50.114167 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-7fbc5785-7ea1-4201-97bf-dbf3fadd92e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:14:50.198876 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:14:50.225310 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=94,keystone:DELETE=4,keystone:INSERT=2 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:14:50.271496 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 834/1676] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:14:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:14:50.274279 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b531fb-85fb-468b-935e-cc1a6deff7f4 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:14:50.283424 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbc5785-7ea1-4201-97bf-dbf3fadd92e1 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:14:50.300868 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b531fb-85fb-468b-935e-cc1a6deff7f4 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:14:50.303290 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d451b4-5d3d-4601-ac1e-d2edffb3060b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:14:50.304049 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d451b4-5d3d-4601-ac1e-d2edffb3060b admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:14:50.304423 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d451b4-5d3d-4601-ac1e-d2edffb3060b admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:14:50.306272 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d451b4-5d3d-4601-ac1e-d2edffb3060b admin admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:14:50.306272 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d451b4-5d3d-4601-ac1e-d2edffb3060b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:14:50.308105 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d451b4-5d3d-4601-ac1e-d2edffb3060b admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:14:50.330061 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d98336c-de3b-4756-aa0d-5af35af1f8b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:14:50.330784 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d98336c-de3b-4756-aa0d-5af35af1f8b7 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:14:50.331149 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d98336c-de3b-4756-aa0d-5af35af1f8b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:14:50.331438 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d98336c-de3b-4756-aa0d-5af35af1f8b7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:14:50.332905 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d98336c-de3b-4756-aa0d-5af35af1f8b7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:14:50.332905 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d98336c-de3b-4756-aa0d-5af35af1f8b7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:14:50.334355 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d98336c-de3b-4756-aa0d-5af35af1f8b7 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:14:50.370409 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 835/1677] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Jul 25 12:14:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:14:50.373669 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d98336c-de3b-4756-aa0d-5af35af1f8b7 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:14:50.379805 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 843/1678] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Jul 25 12:14:50 2022] POST /identity/v3/users => generated 284 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 12:14:50.385395 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d451b4-5d3d-4601-ac1e-d2edffb3060b admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:14:50.404726 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d98336c-de3b-4756-aa0d-5af35af1f8b7 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:14:50.413458 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d451b4-5d3d-4601-ac1e-d2edffb3060b admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:14:50.425004 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef672c9d-e1b6-4496-bbbf-3beaf15184bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:14:50.425701 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef672c9d-e1b6-4496-bbbf-3beaf15184bb admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:14:50.426004 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef672c9d-e1b6-4496-bbbf-3beaf15184bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:14:50.426292 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef672c9d-e1b6-4496-bbbf-3beaf15184bb admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:14:50.427030 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef672c9d-e1b6-4496-bbbf-3beaf15184bb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:14:50.429171 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef672c9d-e1b6-4496-bbbf-3beaf15184bb admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:14:50.440370 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf846b0-f774-4165-b451-d855fdd9a0d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:14:50.441411 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf846b0-f774-4165-b451-d855fdd9a0d8 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:14:50.441731 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf846b0-f774-4165-b451-d855fdd9a0d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:14:50.442030 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf846b0-f774-4165-b451-d855fdd9a0d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:14:50.442800 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf846b0-f774-4165-b451-d855fdd9a0d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:14:50.444534 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf846b0-f774-4165-b451-d855fdd9a0d8 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:14:50.474911 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 844/1679] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Jul 25 12:14:50 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 12:14:50.485279 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf846b0-f774-4165-b451-d855fdd9a0d8 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:14:50.505756 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf846b0-f774-4165-b451-d855fdd9a0d8 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:14:50.520332 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 836/1680] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Jul 25 12:14:50 2022] POST /identity/v3/projects => generated 381 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:14:50.525605 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef672c9d-e1b6-4496-bbbf-3beaf15184bb admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:14:50.533268 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a7f01b-7067-4bdb-9df9-41063cb856f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:14:50.535219 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a7f01b-7067-4bdb-9df9-41063cb856f0 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:14:50.536697 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a7f01b-7067-4bdb-9df9-41063cb856f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:14:50.540705 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a7f01b-7067-4bdb-9df9-41063cb856f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:14:50.543361 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a7f01b-7067-4bdb-9df9-41063cb856f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:14:50.554724 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a7f01b-7067-4bdb-9df9-41063cb856f0 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:14:50.597073 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 845/1681] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Jul 25 12:14:50 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 12:14:50.600590 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a7f01b-7067-4bdb-9df9-41063cb856f0 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:14:50.631482 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a7f01b-7067-4bdb-9df9-41063cb856f0 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:14:50.738099 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef672c9d-e1b6-4496-bbbf-3beaf15184bb admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:14:50.757874 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c967f4-faea-46ed-9d2c-0660c26cf2f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:14:50.758785 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c967f4-faea-46ed-9d2c-0660c26cf2f6 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:14:50.759247 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c967f4-faea-46ed-9d2c-0660c26cf2f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:14:50.759703 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c967f4-faea-46ed-9d2c-0660c26cf2f6 admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:14:50.760560 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c967f4-faea-46ed-9d2c-0660c26cf2f6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:14:50.762661 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c967f4-faea-46ed-9d2c-0660c26cf2f6 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:14:50.812779 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 837/1682] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Jul 25 12:14:50 2022] POST /identity/v3/users => generated 291 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:14:50.816573 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c967f4-faea-46ed-9d2c-0660c26cf2f6 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:14:50.845173 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910011bb-f06c-4e88-afa1-5b8ce4282009 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:14:50.847443 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c967f4-faea-46ed-9d2c-0660c26cf2f6 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:14:50.856491 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910011bb-f06c-4e88-afa1-5b8ce4282009 admin admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:14:50.858948 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910011bb-f06c-4e88-afa1-5b8ce4282009 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:14:50.860233 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910011bb-f06c-4e88-afa1-5b8ce4282009 admin admin] PATH_INFO: `/v3/projects/b06fd349fd574458bb3cccf21be4992d/users/9635467e78bf42edb47faa7ae2bb41eb/roles/472313ae69174578824df0bed6390744` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:14:50.861092 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501292e1-a84e-4bf2-ad23-eac29e1211ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:14:50.861978 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501292e1-a84e-4bf2-ad23-eac29e1211ab admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:14:50.862400 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501292e1-a84e-4bf2-ad23-eac29e1211ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:14:50.862805 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501292e1-a84e-4bf2-ad23-eac29e1211ab admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:14:50.863787 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501292e1-a84e-4bf2-ad23-eac29e1211ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:14:50.865650 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501292e1-a84e-4bf2-ad23-eac29e1211ab admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:14:50.893915 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 838/1683] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Jul 25 12:14:50 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:14:50.898084 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501292e1-a84e-4bf2-ad23-eac29e1211ab admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:14:50.901471 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910011bb-f06c-4e88-afa1-5b8ce4282009 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b06fd349fd574458bb3cccf21be4992d, user_id=9635467e78bf42edb47faa7ae2bb41eb, role_id=472313ae69174578824df0bed6390744)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:14:50.904849 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910011bb-f06c-4e88-afa1-5b8ce4282009 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:14:50.927604 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501292e1-a84e-4bf2-ad23-eac29e1211ab admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:14:50.936056 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 846/1684] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Jul 25 12:14:50 2022] PUT /identity/v3/projects/b06fd349fd574458bb3cccf21be4992d/users/9635467e78bf42edb47faa7ae2bb41eb/roles/472313ae69174578824df0bed6390744 => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:14:50.949098 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9994ba-a614-4807-9f28-d199a8815eba None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:14:50.949542 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9994ba-a614-4807-9f28-d199a8815eba None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:14:50.950034 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9994ba-a614-4807-9f28-d199a8815eba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:14:51.002950 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-9e9994ba-a614-4807-9f28-d199a8815eba None None] MFA Rules not processed for user `9635467e78bf42edb47faa7ae2bb41eb`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:14:51.022219 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-9e9994ba-a614-4807-9f28-d199a8815eba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:14:51.047607 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c65dba1-0c24-4a12-b605-55b52154f212 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:14:51.048360 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c65dba1-0c24-4a12-b605-55b52154f212 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:14:51.048960 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c65dba1-0c24-4a12-b605-55b52154f212 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:14:51.048960 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c65dba1-0c24-4a12-b605-55b52154f212 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:14:51.049649 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c65dba1-0c24-4a12-b605-55b52154f212 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:14:51.056458 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c65dba1-0c24-4a12-b605-55b52154f212 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:14:51.080283 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 839/1685] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Jul 25 12:14:50 2022] GET /identity/v3/roles => generated 1420 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 12:14:51.085046 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c65dba1-0c24-4a12-b605-55b52154f212 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:14:51.103091 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c65dba1-0c24-4a12-b605-55b52154f212 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:14:51.124368 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d8bdf6-8b3c-4c14-ad35-e9263698ec6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:14:51.125300 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d8bdf6-8b3c-4c14-ad35-e9263698ec6f admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:14:51.125594 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d8bdf6-8b3c-4c14-ad35-e9263698ec6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:14:51.125880 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d8bdf6-8b3c-4c14-ad35-e9263698ec6f admin admin] PATH_INFO: `/v3/projects/eac1f4d434044492a139f4d5bad957f5/users/be1a55307e1240478db8ee40a53f4560/roles/472313ae69174578824df0bed6390744` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:14:51.155795 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d8bdf6-8b3c-4c14-ad35-e9263698ec6f admin admin] RBAC: Authorizing `identity:create_grant(project_id=eac1f4d434044492a139f4d5bad957f5, user_id=be1a55307e1240478db8ee40a53f4560, role_id=472313ae69174578824df0bed6390744)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:14:51.158621 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d8bdf6-8b3c-4c14-ad35-e9263698ec6f admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:14:51.184197 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:14:51.190964 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 840/1686] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Jul 25 12:14:51 2022] PUT /identity/v3/projects/eac1f4d434044492a139f4d5bad957f5/users/be1a55307e1240478db8ee40a53f4560/roles/472313ae69174578824df0bed6390744 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 12:14:51.202545 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24acd3c-65b4-45eb-8cbe-3b92d978d200 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:14:51.202841 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24acd3c-65b4-45eb-8cbe-3b92d978d200 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:14:51.203131 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24acd3c-65b4-45eb-8cbe-3b92d978d200 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:14:51.215569 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 847/1687] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:14:50 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 12:14:51.216536 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9994ba-a614-4807-9f28-d199a8815eba None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:14:51.240742 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9994ba-a614-4807-9f28-d199a8815eba None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:14:51.270226 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:14:51.284648 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-f24acd3c-65b4-45eb-8cbe-3b92d978d200 None None] MFA Rules not processed for user `be1a55307e1240478db8ee40a53f4560`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:14:51.286201 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-f24acd3c-65b4-45eb-8cbe-3b92d978d200 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:14:51.378868 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca03e2e7-9b01-4bc7-8b1f-3fc23a4acea6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:14:51.379609 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca03e2e7-9b01-4bc7-8b1f-3fc23a4acea6 admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:14:51.379909 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca03e2e7-9b01-4bc7-8b1f-3fc23a4acea6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:14:51.380194 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca03e2e7-9b01-4bc7-8b1f-3fc23a4acea6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:14:51.380909 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca03e2e7-9b01-4bc7-8b1f-3fc23a4acea6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:14:51.382862 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca03e2e7-9b01-4bc7-8b1f-3fc23a4acea6 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:14:51.430766 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 848/1688] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Jul 25 12:14:51 2022] POST /identity/v3/projects => generated 373 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:14:51.436490 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca03e2e7-9b01-4bc7-8b1f-3fc23a4acea6 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:14:51.478066 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 841/1689] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:14:51 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 276 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 12:14:51.481216 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24acd3c-65b4-45eb-8cbe-3b92d978d200 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:14:51.507519 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24acd3c-65b4-45eb-8cbe-3b92d978d200 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:14:51.604271 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca03e2e7-9b01-4bc7-8b1f-3fc23a4acea6 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:14:51.619012 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ddc0a3-8a85-44d3-ba32-cf2eb5bb2522 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:14:51.619761 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ddc0a3-8a85-44d3-ba32-cf2eb5bb2522 admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:14:51.620051 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ddc0a3-8a85-44d3-ba32-cf2eb5bb2522 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:14:51.620335 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ddc0a3-8a85-44d3-ba32-cf2eb5bb2522 admin admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:14:51.624573 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ddc0a3-8a85-44d3-ba32-cf2eb5bb2522 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:14:51.629665 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ddc0a3-8a85-44d3-ba32-cf2eb5bb2522 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:14:51.651593 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979aa6dc-d56f-43aa-9247-5402244d42c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:14:51.652327 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979aa6dc-d56f-43aa-9247-5402244d42c3 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:14:51.652614 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979aa6dc-d56f-43aa-9247-5402244d42c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:14:51.652902 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979aa6dc-d56f-43aa-9247-5402244d42c3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:14:51.654617 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979aa6dc-d56f-43aa-9247-5402244d42c3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:14:51.658040 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979aa6dc-d56f-43aa-9247-5402244d42c3 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:14:51.684064 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 849/1690] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Jul 25 12:14:51 2022] POST /identity/v3/users => generated 293 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:14:51.685212 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ddc0a3-8a85-44d3-ba32-cf2eb5bb2522 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:14:51.707219 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ddc0a3-8a85-44d3-ba32-cf2eb5bb2522 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:14:51.722677 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25688f54-123d-48a7-95a8-d655bf93fc41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:14:51.726969 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 842/1691] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Jul 25 12:14:51 2022] POST /identity/v3/projects => generated 377 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:14:51.727772 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25688f54-123d-48a7-95a8-d655bf93fc41 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:14:51.728657 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979aa6dc-d56f-43aa-9247-5402244d42c3 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:14:51.729235 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25688f54-123d-48a7-95a8-d655bf93fc41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:14:51.729523 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25688f54-123d-48a7-95a8-d655bf93fc41 admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:14:51.730517 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25688f54-123d-48a7-95a8-d655bf93fc41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:14:51.733781 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25688f54-123d-48a7-95a8-d655bf93fc41 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:14:51.767683 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 850/1692] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Jul 25 12:14:51 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:14:51.768470 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25688f54-123d-48a7-95a8-d655bf93fc41 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:14:51.793150 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25688f54-123d-48a7-95a8-d655bf93fc41 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:14:51.885398 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979aa6dc-d56f-43aa-9247-5402244d42c3 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:14:51.899790 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f3a762-d42c-4c8f-8210-f55e340e8278 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:14:51.900497 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f3a762-d42c-4c8f-8210-f55e340e8278 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:14:51.900780 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f3a762-d42c-4c8f-8210-f55e340e8278 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:14:51.901120 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f3a762-d42c-4c8f-8210-f55e340e8278 admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:14:51.902292 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f3a762-d42c-4c8f-8210-f55e340e8278 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:14:51.906608 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f3a762-d42c-4c8f-8210-f55e340e8278 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:14:51.943352 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a3d45a-5001-45b6-8aa6-61097f546aa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:14:51.944588 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a3d45a-5001-45b6-8aa6-61097f546aa5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:14:51.944806 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a3d45a-5001-45b6-8aa6-61097f546aa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:14:51.944980 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a3d45a-5001-45b6-8aa6-61097f546aa5 admin admin] PATH_INFO: `/v3/projects/d25ef4d0d1b940c895dede7abebd1468/users/4947ee15b2c04c8fb01ed2940d12a2bb/roles/a7d20885b6d94928a34b3cc5b3a00006` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:14:51.973731 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 843/1693] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Jul 25 12:14:51 2022] POST /identity/v3/users => generated 295 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:14:51.980068 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f3a762-d42c-4c8f-8210-f55e340e8278 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:14:51.989923 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:14:52.002593 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f3a762-d42c-4c8f-8210-f55e340e8278 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:14:52.006922 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a3d45a-5001-45b6-8aa6-61097f546aa5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d25ef4d0d1b940c895dede7abebd1468, user_id=4947ee15b2c04c8fb01ed2940d12a2bb, role_id=a7d20885b6d94928a34b3cc5b3a00006)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:14:52.011350 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a3d45a-5001-45b6-8aa6-61097f546aa5 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:14:52.018792 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59a7c06-73e6-49c7-8482-291b526d0626 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:14:52.019499 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59a7c06-73e6-49c7-8482-291b526d0626 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:14:52.019862 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59a7c06-73e6-49c7-8482-291b526d0626 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:14:52.020153 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59a7c06-73e6-49c7-8482-291b526d0626 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:14:52.020912 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59a7c06-73e6-49c7-8482-291b526d0626 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:14:52.023735 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59a7c06-73e6-49c7-8482-291b526d0626 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:14:52.038513 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:14:52.047564 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 851/1694] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Jul 25 12:14:51 2022] PUT /identity/v3/projects/d25ef4d0d1b940c895dede7abebd1468/users/4947ee15b2c04c8fb01ed2940d12a2bb/roles/a7d20885b6d94928a34b3cc5b3a00006 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:14:52.049364 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a3d45a-5001-45b6-8aa6-61097f546aa5 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:14:52.057086 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 844/1695] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Jul 25 12:14:51 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:14:52.058031 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59a7c06-73e6-49c7-8482-291b526d0626 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:14:52.081540 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59a7c06-73e6-49c7-8482-291b526d0626 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:14:52.194078 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a3d45a-5001-45b6-8aa6-61097f546aa5 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:14:52.211171 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec3e515-5feb-47b0-9317-e4cb48d44d45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:14:52.215500 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec3e515-5feb-47b0-9317-e4cb48d44d45 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:14:52.215852 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec3e515-5feb-47b0-9317-e4cb48d44d45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:14:52.216363 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec3e515-5feb-47b0-9317-e4cb48d44d45 admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:14:52.216903 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec3e515-5feb-47b0-9317-e4cb48d44d45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:14:52.218554 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec3e515-5feb-47b0-9317-e4cb48d44d45 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:14:52.228957 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d932b5-f072-42a5-96a5-85a0b5179e3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:14:52.233419 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d932b5-f072-42a5-96a5-85a0b5179e3b admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:14:52.233830 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d932b5-f072-42a5-96a5-85a0b5179e3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:14:52.234199 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d932b5-f072-42a5-96a5-85a0b5179e3b admin admin] PATH_INFO: `/v3/projects/3a49657f567c4de5a4a546ddd7e587ef/users/c6887c60d1cd4a91a68c976d6f1fdafa/roles/a7d20885b6d94928a34b3cc5b3a00006` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:14:52.261095 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 852/1696] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Jul 25 12:14:52 2022] GET /identity/v3/roles => generated 1420 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:14:52.267377 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec3e515-5feb-47b0-9317-e4cb48d44d45 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:14:52.299788 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d932b5-f072-42a5-96a5-85a0b5179e3b admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a49657f567c4de5a4a546ddd7e587ef, user_id=c6887c60d1cd4a91a68c976d6f1fdafa, role_id=a7d20885b6d94928a34b3cc5b3a00006)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:14:52.308148 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d932b5-f072-42a5-96a5-85a0b5179e3b admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:14:52.314019 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec3e515-5feb-47b0-9317-e4cb48d44d45 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:14:52.329470 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bba209-14f2-451e-acd0-1599f51e3b8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:14:52.336515 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bba209-14f2-451e-acd0-1599f51e3b8f admin admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:14:52.337419 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bba209-14f2-451e-acd0-1599f51e3b8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:14:52.339923 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bba209-14f2-451e-acd0-1599f51e3b8f admin admin] PATH_INFO: `/v3/projects/d25ef4d0d1b940c895dede7abebd1468/users/4947ee15b2c04c8fb01ed2940d12a2bb/roles/a7d20885b6d94928a34b3cc5b3a00006` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:14:52.346809 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19bba209-14f2-451e-acd0-1599f51e3b8f admin admin] RBAC: Authorizing `identity:create_grant(project_id=d25ef4d0d1b940c895dede7abebd1468, user_id=4947ee15b2c04c8fb01ed2940d12a2bb, role_id=a7d20885b6d94928a34b3cc5b3a00006)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:14:52.353731 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 845/1697] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Jul 25 12:14:52 2022] PUT /identity/v3/projects/3a49657f567c4de5a4a546ddd7e587ef/users/c6887c60d1cd4a91a68c976d6f1fdafa/roles/a7d20885b6d94928a34b3cc5b3a00006 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:14:52.356415 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d932b5-f072-42a5-96a5-85a0b5179e3b admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:14:52.362748 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19bba209-14f2-451e-acd0-1599f51e3b8f admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:14:52.416822 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 853/1698] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Jul 25 12:14:52 2022] PUT /identity/v3/projects/d25ef4d0d1b940c895dede7abebd1468/users/4947ee15b2c04c8fb01ed2940d12a2bb/roles/a7d20885b6d94928a34b3cc5b3a00006 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:14:52.418187 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e4fb6c-3709-48cc-a6fc-39298b5c8689 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:14:52.418488 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e4fb6c-3709-48cc-a6fc-39298b5c8689 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:14:52.418761 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e4fb6c-3709-48cc-a6fc-39298b5c8689 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:14:52.492951 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-45e4fb6c-3709-48cc-a6fc-39298b5c8689 None None] MFA Rules not processed for user `4947ee15b2c04c8fb01ed2940d12a2bb`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:14:52.498186 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-45e4fb6c-3709-48cc-a6fc-39298b5c8689 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:14:52.586668 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d932b5-f072-42a5-96a5-85a0b5179e3b admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:14:52.714835 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 854/1699] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:14:52 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 298 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:14:52.725410 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffcde69-fe88-4ad2-9ba7-bbd478880d25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:14:52.726541 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffcde69-fe88-4ad2-9ba7-bbd478880d25 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:14:52.727199 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffcde69-fe88-4ad2-9ba7-bbd478880d25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:14:52.727588 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffcde69-fe88-4ad2-9ba7-bbd478880d25 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:14:52.728934 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffcde69-fe88-4ad2-9ba7-bbd478880d25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:14:52.730679 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e4fb6c-3709-48cc-a6fc-39298b5c8689 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:14:52.731202 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffcde69-fe88-4ad2-9ba7-bbd478880d25 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:14:52.732318 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-45e4fb6c-3709-48cc-a6fc-39298b5c8689 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:14:52.744432 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 846/1700] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Jul 25 12:14:52 2022] GET /identity/v3/roles => generated 1420 bytes in 390 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:14:52.748921 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffcde69-fe88-4ad2-9ba7-bbd478880d25 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:14:52.765592 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffcde69-fe88-4ad2-9ba7-bbd478880d25 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:14:52.784993 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde8e67f-e943-497d-b008-d31f05aa7014 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:14:52.786398 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde8e67f-e943-497d-b008-d31f05aa7014 admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:14:52.786784 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde8e67f-e943-497d-b008-d31f05aa7014 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:14:52.787164 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde8e67f-e943-497d-b008-d31f05aa7014 admin admin] PATH_INFO: `/v3/projects/3a49657f567c4de5a4a546ddd7e587ef/users/c6887c60d1cd4a91a68c976d6f1fdafa/roles/a7d20885b6d94928a34b3cc5b3a00006` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:14:52.790834 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde8e67f-e943-497d-b008-d31f05aa7014 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a49657f567c4de5a4a546ddd7e587ef, user_id=c6887c60d1cd4a91a68c976d6f1fdafa, role_id=a7d20885b6d94928a34b3cc5b3a00006)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:14:52.795516 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde8e67f-e943-497d-b008-d31f05aa7014 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:14:52.812769 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 847/1701] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Jul 25 12:14:52 2022] PUT /identity/v3/projects/3a49657f567c4de5a4a546ddd7e587ef/users/c6887c60d1cd4a91a68c976d6f1fdafa/roles/a7d20885b6d94928a34b3cc5b3a00006 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:14:52.823740 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e83a79-6f8e-4f5f-968a-c3e65801e134 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:14:52.824274 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e83a79-6f8e-4f5f-968a-c3e65801e134 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:14:52.825640 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e83a79-6f8e-4f5f-968a-c3e65801e134 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:14:52.851418 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:14:52.889186 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e4fb6c-3709-48cc-a6fc-39298b5c8689 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:14:52.897264 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-b9e83a79-6f8e-4f5f-968a-c3e65801e134 None None] MFA Rules not processed for user `c6887c60d1cd4a91a68c976d6f1fdafa`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:14:52.898526 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-b9e83a79-6f8e-4f5f-968a-c3e65801e134 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:14:52.943827 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:14:53.014124 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093741d9-cde8-4b65-a564-13af016f2289 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f89ce6f0cf04373a414d927fa0f8a7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:14:53.015113 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093741d9-cde8-4b65-a564-13af016f2289 service cinder] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:14:53.016073 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093741d9-cde8-4b65-a564-13af016f2289 service cinder] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:14:53.016598 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093741d9-cde8-4b65-a564-13af016f2289 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:14:53.083170 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 848/1702] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:14:52 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 12:14:53.132360 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093741d9-cde8-4b65-a564-13af016f2289 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:14:53.149921 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093741d9-cde8-4b65-a564-13af016f2289 service cinder] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:14:53.170505 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 855/1703] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:14:52 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 441 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:14:53.210347 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e83a79-6f8e-4f5f-968a-c3e65801e134 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:14:53.214963 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-b9e83a79-6f8e-4f5f-968a-c3e65801e134 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:14:53.240468 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e83a79-6f8e-4f5f-968a-c3e65801e134 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:14:53.253916 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766a1d84-1008-4654-a7ea-1cab90f1a50e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f89ce6f0cf04373a414d927fa0f8a7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:14:53.254598 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766a1d84-1008-4654-a7ea-1cab90f1a50e service cinder] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:14:53.254893 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766a1d84-1008-4654-a7ea-1cab90f1a50e service cinder] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:14:53.255180 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766a1d84-1008-4654-a7ea-1cab90f1a50e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:14:53.278738 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766a1d84-1008-4654-a7ea-1cab90f1a50e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:14:53.284889 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766a1d84-1008-4654-a7ea-1cab90f1a50e service cinder] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:14:53.306469 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 849/1704] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:14:53 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:14:53.470970 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6594d5-f707-4af4-a9a2-4ff870128494 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:14:53.471490 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6594d5-f707-4af4-a9a2-4ff870128494 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:14:53.471617 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6594d5-f707-4af4-a9a2-4ff870128494 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:14:53.474619 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766a1d84-1008-4654-a7ea-1cab90f1a50e service cinder] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:14:53.486019 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766a1d84-1008-4654-a7ea-1cab90f1a50e service cinder] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:14:53.504116 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321a4d9d-f5f7-46d8-8017-4450ae7aa6cb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f89ce6f0cf04373a414d927fa0f8a7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:14:53.504812 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321a4d9d-f5f7-46d8-8017-4450ae7aa6cb service cinder] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:14:53.505415 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321a4d9d-f5f7-46d8-8017-4450ae7aa6cb service cinder] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:14:53.505415 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321a4d9d-f5f7-46d8-8017-4450ae7aa6cb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:14:53.544546 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-6f6594d5-f707-4af4-a9a2-4ff870128494 None None] MFA Rules not processed for user `a766d17b116f498fab07efb69c94e49e`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:14:53.550237 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-6f6594d5-f707-4af4-a9a2-4ff870128494 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:14:53.708481 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321a4d9d-f5f7-46d8-8017-4450ae7aa6cb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:14:53.710339 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321a4d9d-f5f7-46d8-8017-4450ae7aa6cb service cinder] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:14:53.740869 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 856/1705] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:14:53 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:14:53.741797 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 850/1706] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:14:53 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 268 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 12:14:53.752814 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94296d01-d6ad-42cf-bb42-43da7397d5bb None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:14:53.753533 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94296d01-d6ad-42cf-bb42-43da7397d5bb None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:14:53.754152 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94296d01-d6ad-42cf-bb42-43da7397d5bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:14:53.804620 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-94296d01-d6ad-42cf-bb42-43da7397d5bb None None] MFA Rules not processed for user `a766d17b116f498fab07efb69c94e49e`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:14:53.814213 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-94296d01-d6ad-42cf-bb42-43da7397d5bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:14:53.833237 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e14e16-b353-4ae6-8ea5-02d2207b1b9f None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:14:53.833837 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e14e16-b353-4ae6-8ea5-02d2207b1b9f None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:14:53.834202 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e14e16-b353-4ae6-8ea5-02d2207b1b9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:14:53.844936 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 857/1707] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:14:53 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:14:53.858698 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94296d01-d6ad-42cf-bb42-43da7397d5bb None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:14:53.860862 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-94296d01-d6ad-42cf-bb42-43da7397d5bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:14:53.887378 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-74e14e16-b353-4ae6-8ea5-02d2207b1b9f None None] MFA Rules not processed for user `d2d6f10a6fe841e5b1d12bce78f97d91`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:14:53.889303 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-74e14e16-b353-4ae6-8ea5-02d2207b1b9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:14:54.016025 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:14:54.049424 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 851/1708] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:14:53 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 12:14:54.054175 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94296d01-d6ad-42cf-bb42-43da7397d5bb None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:14:54.072043 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ddebee-7d0f-4bba-bade-1315a65c8c2c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f67142a1193d47c7bd33d7040268233c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:14:54.072919 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ddebee-7d0f-4bba-bade-1315a65c8c2c service nova] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:14:54.073210 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ddebee-7d0f-4bba-bade-1315a65c8c2c service nova] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:14:54.073501 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ddebee-7d0f-4bba-bade-1315a65c8c2c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:14:54.108891 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ddebee-7d0f-4bba-bade-1315a65c8c2c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:14:54.113672 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ddebee-7d0f-4bba-bade-1315a65c8c2c service nova] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:14:54.128299 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 858/1709] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:14:53 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 270 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 12:14:54.350526 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e14e16-b353-4ae6-8ea5-02d2207b1b9f None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:14:54.410005 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:14:54.550653 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e14e16-b353-4ae6-8ea5-02d2207b1b9f None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:14:54.555254 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9882c937-07d3-45d1-950e-883119798942 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:14:54.557118 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9882c937-07d3-45d1-950e-883119798942 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:14:54.557556 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9882c937-07d3-45d1-950e-883119798942 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:14:54.557844 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9882c937-07d3-45d1-950e-883119798942 admin admin] PATH_INFO: `/v3/users/9635467e78bf42edb47faa7ae2bb41eb` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:14:54.561526 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9882c937-07d3-45d1-950e-883119798942 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9635467e78bf42edb47faa7ae2bb41eb)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:14:54.563569 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9882c937-07d3-45d1-950e-883119798942 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:14:54.685055 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-9882c937-07d3-45d1-950e-883119798942 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9635467e78bf42edb47faa7ae2bb41eb'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:14:54.706725 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-9882c937-07d3-45d1-950e-883119798942 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9635467e78bf42edb47faa7ae2bb41eb'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:14:54.730613 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-9882c937-07d3-45d1-950e-883119798942 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9635467e78bf42edb47faa7ae2bb41eb'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:14:54.761948 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-9882c937-07d3-45d1-950e-883119798942 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9635467e78bf42edb47faa7ae2bb41eb'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:14:54.762885 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-9882c937-07d3-45d1-950e-883119798942 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9635467e78bf42edb47faa7ae2bb41eb'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:14:54.799263 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 852/1710] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Jul 25 12:14:54 2022] DELETE /identity/v3/users/9635467e78bf42edb47faa7ae2bb41eb => generated 0 bytes in 447 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:14:54.801631 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf54390e-1bfa-43e1-a178-f60bfe7d8af3 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:14:54.802115 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf54390e-1bfa-43e1-a178-f60bfe7d8af3 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:14:54.802293 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf54390e-1bfa-43e1-a178-f60bfe7d8af3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:14:54.835285 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-cf54390e-1bfa-43e1-a178-f60bfe7d8af3 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:14:54.836581 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-cf54390e-1bfa-43e1-a178-f60bfe7d8af3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:14:54.983594 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 859/1711] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:14:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:14:55.009474 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9882c937-07d3-45d1-950e-883119798942 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:14:55.011073 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-9882c937-07d3-45d1-950e-883119798942 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:14:55.070566 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae83ee1-07f0-4bbe-93d9-0391cac904c9 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:14:55.070878 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae83ee1-07f0-4bbe-93d9-0391cac904c9 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:14:55.071181 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae83ee1-07f0-4bbe-93d9-0391cac904c9 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:14:55.072820 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 860/1712] 10.222.0.42 () {54 vars in 912 bytes} [Mon Jul 25 12:14:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 12:14:55.080169 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51601394-9364-4ed1-a896-8793557cdf54 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:14:55.080472 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51601394-9364-4ed1-a896-8793557cdf54 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:14:55.080750 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51601394-9364-4ed1-a896-8793557cdf54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:14:55.140493 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9882c937-07d3-45d1-950e-883119798942 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:14:55.166614 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6037a0-4ed0-40a4-9b51-cc607a380c4a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51733781201243e8abf9eca39b207750', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:14:55.167690 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6037a0-4ed0-40a4-9b51-cc607a380c4a service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:14:55.169673 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6037a0-4ed0-40a4-9b51-cc607a380c4a service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:14:55.170323 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6037a0-4ed0-40a4-9b51-cc607a380c4a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:14:55.174216 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-51601394-9364-4ed1-a896-8793557cdf54 None None] MFA Rules not processed for user `51733781201243e8abf9eca39b207750`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:14:55.176525 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-51601394-9364-4ed1-a896-8793557cdf54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:14:55.191282 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 861/1713] 10.222.0.42 () {58 vars in 1009 bytes} [Mon Jul 25 12:14:55 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:14:55.202305 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6037a0-4ed0-40a4-9b51-cc607a380c4a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:14:55.202912 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51601394-9364-4ed1-a896-8793557cdf54 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:14:55.204785 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-51601394-9364-4ed1-a896-8793557cdf54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:14:55.204903 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6037a0-4ed0-40a4-9b51-cc607a380c4a service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:14:55.218466 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51601394-9364-4ed1-a896-8793557cdf54 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:14:55.231037 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 853/1714] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:14:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 12:14:55.232819 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fdbbf5-38c7-4904-9d8d-ded97fe564ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51733781201243e8abf9eca39b207750', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:14:55.233516 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fdbbf5-38c7-4904-9d8d-ded97fe564ba service neutron] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:14:55.233810 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fdbbf5-38c7-4904-9d8d-ded97fe564ba service neutron] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:14:55.234092 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fdbbf5-38c7-4904-9d8d-ded97fe564ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:14:55.244512 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fdbbf5-38c7-4904-9d8d-ded97fe564ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:14:55.245957 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fdbbf5-38c7-4904-9d8d-ded97fe564ba service neutron] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:14:55.262630 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 862/1715] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:14:55 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 12:14:55.582762 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8e6343-aaf6-4999-8b76-9add9959a7cd None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:14:55.583410 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8e6343-aaf6-4999-8b76-9add9959a7cd None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:14:55.584102 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8e6343-aaf6-4999-8b76-9add9959a7cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:14:55.677306 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-4a8e6343-aaf6-4999-8b76-9add9959a7cd None None] MFA Rules not processed for user `c5b98bbcc2d640de837d141f83dfe924`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:14:55.682444 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-4a8e6343-aaf6-4999-8b76-9add9959a7cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:14:55.698626 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678d048b-8450-4fc4-999d-5f39283c423c None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:14:55.699135 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678d048b-8450-4fc4-999d-5f39283c423c None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:14:55.699572 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678d048b-8450-4fc4-999d-5f39283c423c None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:14:55.701116 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 863/1716] 10.222.0.42 () {54 vars in 912 bytes} [Mon Jul 25 12:14:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 12:14:55.710015 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8961ef4-ced7-4895-bcd6-a98cdaa20e18 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:14:55.711020 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8961ef4-ced7-4895-bcd6-a98cdaa20e18 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:14:55.711020 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8961ef4-ced7-4895-bcd6-a98cdaa20e18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:14:55.784540 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-c8961ef4-ced7-4895-bcd6-a98cdaa20e18 None None] MFA Rules not processed for user `51733781201243e8abf9eca39b207750`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:14:55.786282 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-c8961ef4-ced7-4895-bcd6-a98cdaa20e18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:14:55.796567 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 864/1717] 10.222.0.42 () {58 vars in 1009 bytes} [Mon Jul 25 12:14:55 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:14:55.820395 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:14:55.884891 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8961ef4-ced7-4895-bcd6-a98cdaa20e18 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:14:55.897873 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8961ef4-ced7-4895-bcd6-a98cdaa20e18 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:14:55.912730 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46c16b4-1534-415e-923b-783cda927c08 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51733781201243e8abf9eca39b207750', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:14:55.913698 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46c16b4-1534-415e-923b-783cda927c08 service neutron] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:14:55.914184 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46c16b4-1534-415e-923b-783cda927c08 service neutron] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:14:55.915447 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46c16b4-1534-415e-923b-783cda927c08 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:14:55.939948 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46c16b4-1534-415e-923b-783cda927c08 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:14:55.952971 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 854/1718] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:14:55 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 371 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:14:55.954836 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46c16b4-1534-415e-923b-783cda927c08 service neutron] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:14:55.975756 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 865/1719] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:14:55 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:14:56.042011 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57d7c66-b17c-4156-b290-a85e38948ad8 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:14:56.042725 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57d7c66-b17c-4156-b290-a85e38948ad8 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:14:56.043340 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57d7c66-b17c-4156-b290-a85e38948ad8 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:14:56.045071 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 855/1720] 10.222.0.42 () {54 vars in 961 bytes} [Mon Jul 25 12:14:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:14:56.056521 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef92d58-c9ce-4ade-8a31-78474b5a30e0 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:14:56.056816 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef92d58-c9ce-4ade-8a31-78474b5a30e0 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:14:56.057088 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef92d58-c9ce-4ade-8a31-78474b5a30e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:14:56.080351 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-4ef92d58-c9ce-4ade-8a31-78474b5a30e0 None None] MFA Rules not processed for user `8012afab6cac4358ab878732e58d0689`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:14:56.081629 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-4ef92d58-c9ce-4ade-8a31-78474b5a30e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:14:56.170023 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57d7c66-b17c-4156-b290-a85e38948ad8 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:14:56.183848 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-e57d7c66-b17c-4156-b290-a85e38948ad8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:14:56.191833 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 866/1721] 10.222.0.42 () {58 vars in 1058 bytes} [Mon Jul 25 12:14:56 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:14:56.194340 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57d7c66-b17c-4156-b290-a85e38948ad8 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:14:56.196957 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef92d58-c9ce-4ade-8a31-78474b5a30e0 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:14:56.198699 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-4ef92d58-c9ce-4ade-8a31-78474b5a30e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:14:56.200818 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6322fbc6-5658-4899-b272-f6f33816f8ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:14:56.201814 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6322fbc6-5658-4899-b272-f6f33816f8ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:14:56.202099 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6322fbc6-5658-4899-b272-f6f33816f8ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:14:56.202430 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6322fbc6-5658-4899-b272-f6f33816f8ba admin admin] PATH_INFO: `/v3/projects/b06fd349fd574458bb3cccf21be4992d` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:14:56.203663 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6322fbc6-5658-4899-b272-f6f33816f8ba admin admin] RBAC: Authorizing `identity:delete_project(project_id=b06fd349fd574458bb3cccf21be4992d)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:14:56.205879 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6322fbc6-5658-4899-b272-f6f33816f8ba admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:14:56.256324 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:14:56.287214 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-6322fbc6-5658-4899-b272-f6f33816f8ba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b06fd349fd574458bb3cccf21be4992d', 'request_id': 'req-6322fbc6-5658-4899-b272-f6f33816f8ba'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:14:56.299340 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-6322fbc6-5658-4899-b272-f6f33816f8ba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b06fd349fd574458bb3cccf21be4992d', 'request_id': 'req-6322fbc6-5658-4899-b272-f6f33816f8ba'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:14:56.309653 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-6322fbc6-5658-4899-b272-f6f33816f8ba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b06fd349fd574458bb3cccf21be4992d', 'request_id': 'req-6322fbc6-5658-4899-b272-f6f33816f8ba'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:14:56.313979 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef92d58-c9ce-4ade-8a31-78474b5a30e0 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:14:56.327320 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-6322fbc6-5658-4899-b272-f6f33816f8ba admin admin] The token cache is being invalidate because project b06fd349fd574458bb3cccf21be4992d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 12:14:56.327804 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-6322fbc6-5658-4899-b272-f6f33816f8ba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:14:56.328665 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-6322fbc6-5658-4899-b272-f6f33816f8ba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:14:56.330992 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 856/1722] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Jul 25 12:14:56 2022] DELETE /identity/v3/projects/b06fd349fd574458bb3cccf21be4992d => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:14:56.333447 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6322fbc6-5658-4899-b272-f6f33816f8ba admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:14:56.335806 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-6322fbc6-5658-4899-b272-f6f33816f8ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:14:56.429736 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d1e540-0f97-4b6d-9e5c-cdf8eae62f50 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f67142a1193d47c7bd33d7040268233c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:14:56.431359 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d1e540-0f97-4b6d-9e5c-cdf8eae62f50 service nova] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:14:56.431910 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d1e540-0f97-4b6d-9e5c-cdf8eae62f50 service nova] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:14:56.432189 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d1e540-0f97-4b6d-9e5c-cdf8eae62f50 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:14:56.434084 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-15d1e540-0f97-4b6d-9e5c-cdf8eae62f50 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:14:56.458612 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6322fbc6-5658-4899-b272-f6f33816f8ba admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:14:56.465254 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99de1129-3c15-4dd4-a9eb-91406c5c4796 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8012afab6cac4358ab878732e58d0689', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:14:56.466556 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99de1129-3c15-4dd4-a9eb-91406c5c4796 service placement] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:14:56.467110 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99de1129-3c15-4dd4-a9eb-91406c5c4796 service placement] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:14:56.467642 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99de1129-3c15-4dd4-a9eb-91406c5c4796 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:14:56.470384 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-99de1129-3c15-4dd4-a9eb-91406c5c4796 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:14:56.489458 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99de1129-3c15-4dd4-a9eb-91406c5c4796 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:14:56.491859 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99de1129-3c15-4dd4-a9eb-91406c5c4796 service placement] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:14:56.515230 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 857/1723] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:14:56 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:14:56.527280 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99de1129-3c15-4dd4-a9eb-91406c5c4796 service placement] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:14:56.531023 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-99de1129-3c15-4dd4-a9eb-91406c5c4796 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:14:56.544261 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d1e540-0f97-4b6d-9e5c-cdf8eae62f50 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:14:56.546017 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d1e540-0f97-4b6d-9e5c-cdf8eae62f50 service nova] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:14:56.558910 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 867/1724] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:14:56 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 12:14:56.560887 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87317273-f951-4286-a2bb-441bd65c24ee None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:14:56.561290 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87317273-f951-4286-a2bb-441bd65c24ee None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:14:56.561812 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87317273-f951-4286-a2bb-441bd65c24ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:14:56.597085 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-87317273-f951-4286-a2bb-441bd65c24ee None None] MFA Rules not processed for user `3848eae5b8df48eb9816ba0d7485f7c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:14:56.598671 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-87317273-f951-4286-a2bb-441bd65c24ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:14:56.685643 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:14:56.714504 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:14:56.732519 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99de1129-3c15-4dd4-a9eb-91406c5c4796 service placement] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:14:56.739060 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52166093-3a10-4593-9522-47614d7f1008 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:14:56.740128 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52166093-3a10-4593-9522-47614d7f1008 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:14:56.741576 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52166093-3a10-4593-9522-47614d7f1008 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:14:56.742711 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52166093-3a10-4593-9522-47614d7f1008 admin admin] PATH_INFO: `/v3/users/4947ee15b2c04c8fb01ed2940d12a2bb` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:14:56.748206 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52166093-3a10-4593-9522-47614d7f1008 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4947ee15b2c04c8fb01ed2940d12a2bb)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:14:56.757467 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52166093-3a10-4593-9522-47614d7f1008 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:14:56.776071 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 868/1725] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:14:56 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:14:56.796201 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87317273-f951-4286-a2bb-441bd65c24ee None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:14:56.801335 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-87317273-f951-4286-a2bb-441bd65c24ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:14:56.863455 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87317273-f951-4286-a2bb-441bd65c24ee None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:14:56.896096 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5fdc59-d0ec-4f2b-b119-3acdbaa6382b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f67142a1193d47c7bd33d7040268233c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:14:56.897156 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5fdc59-d0ec-4f2b-b119-3acdbaa6382b service nova] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:14:56.897244 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5fdc59-d0ec-4f2b-b119-3acdbaa6382b service nova] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:14:56.897503 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5fdc59-d0ec-4f2b-b119-3acdbaa6382b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:14:56.902032 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-52166093-3a10-4593-9522-47614d7f1008 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4947ee15b2c04c8fb01ed2940d12a2bb'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:14:56.909292 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5fdc59-d0ec-4f2b-b119-3acdbaa6382b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:14:56.911802 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5fdc59-d0ec-4f2b-b119-3acdbaa6382b service nova] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:14:56.914717 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-52166093-3a10-4593-9522-47614d7f1008 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4947ee15b2c04c8fb01ed2940d12a2bb'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:14:56.932904 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-52166093-3a10-4593-9522-47614d7f1008 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4947ee15b2c04c8fb01ed2940d12a2bb'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:14:56.937079 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 869/1726] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:14:56 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:14:56.957052 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-52166093-3a10-4593-9522-47614d7f1008 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4947ee15b2c04c8fb01ed2940d12a2bb'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:14:56.958439 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-52166093-3a10-4593-9522-47614d7f1008 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4947ee15b2c04c8fb01ed2940d12a2bb'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:14:56.971090 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 858/1727] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Jul 25 12:14:56 2022] DELETE /identity/v3/users/4947ee15b2c04c8fb01ed2940d12a2bb => generated 0 bytes in 455 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:14:58.014993 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905e5023-5986-4315-a466-86903731e096 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:14:58.015949 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905e5023-5986-4315-a466-86903731e096 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:14:58.016366 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905e5023-5986-4315-a466-86903731e096 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:14:58.094396 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-905e5023-5986-4315-a466-86903731e096 None None] MFA Rules not processed for user `a766d17b116f498fab07efb69c94e49e`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:14:58.099162 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-905e5023-5986-4315-a466-86903731e096 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:14:58.345843 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90057b2-8eee-4469-9033-9104b5d8a2ae None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:14:58.346509 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90057b2-8eee-4469-9033-9104b5d8a2ae None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:14:58.346585 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90057b2-8eee-4469-9033-9104b5d8a2ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:14:58.363104 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 870/1728] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:14:58 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 349 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:14:58.432613 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-a90057b2-8eee-4469-9033-9104b5d8a2ae None None] MFA Rules not processed for user `89a96a9ce9db422f96927b7f59606998`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:14:58.450676 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-a90057b2-8eee-4469-9033-9104b5d8a2ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:14:58.756834 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 859/1729] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:14:58 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 412 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:14:58.757291 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41939ab-0850-4bd6-b8b9-4bbe684dbefd None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:14:58.757594 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41939ab-0850-4bd6-b8b9-4bbe684dbefd None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:14:58.757889 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41939ab-0850-4bd6-b8b9-4bbe684dbefd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:14:58.859045 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-b41939ab-0850-4bd6-b8b9-4bbe684dbefd None None] MFA Rules not processed for user `89a96a9ce9db422f96927b7f59606998`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:14:58.862794 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-b41939ab-0850-4bd6-b8b9-4bbe684dbefd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:14:58.898382 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 871/1730] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:14:58 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:14:59.263733 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90057b2-8eee-4469-9033-9104b5d8a2ae None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:14:59.265555 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-a90057b2-8eee-4469-9033-9104b5d8a2ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:14:59.367216 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41939ab-0850-4bd6-b8b9-4bbe684dbefd None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:14:59.370758 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-b41939ab-0850-4bd6-b8b9-4bbe684dbefd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:14:59.451657 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:14:59.522153 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90057b2-8eee-4469-9033-9104b5d8a2ae None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:14:59.534946 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd51cbb-1c39-4d20-9b49-3cebc22f548a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f67142a1193d47c7bd33d7040268233c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:14:59.535949 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd51cbb-1c39-4d20-9b49-3cebc22f548a service nova] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:14:59.536242 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd51cbb-1c39-4d20-9b49-3cebc22f548a service nova] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:14:59.536804 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd51cbb-1c39-4d20-9b49-3cebc22f548a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:14:59.548429 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd51cbb-1c39-4d20-9b49-3cebc22f548a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:14:59.550769 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd51cbb-1c39-4d20-9b49-3cebc22f548a service nova] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:14:59.575752 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41939ab-0850-4bd6-b8b9-4bbe684dbefd None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:14:59.575958 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 860/1731] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:14:59 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:14:59.590993 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d19363b-00fa-4d49-8da3-87e0d2943ddf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:14:59.592130 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d19363b-00fa-4d49-8da3-87e0d2943ddf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:14:59.592527 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d19363b-00fa-4d49-8da3-87e0d2943ddf admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:14:59.592924 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d19363b-00fa-4d49-8da3-87e0d2943ddf admin admin] PATH_INFO: `/v3/projects/d25ef4d0d1b940c895dede7abebd1468` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:14:59.596153 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d19363b-00fa-4d49-8da3-87e0d2943ddf admin admin] RBAC: Authorizing `identity:delete_project(project_id=d25ef4d0d1b940c895dede7abebd1468)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:14:59.603075 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d19363b-00fa-4d49-8da3-87e0d2943ddf admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:14:59.784941 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-6d19363b-00fa-4d49-8da3-87e0d2943ddf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd25ef4d0d1b940c895dede7abebd1468', 'request_id': 'req-6d19363b-00fa-4d49-8da3-87e0d2943ddf'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:14:59.816779 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-6d19363b-00fa-4d49-8da3-87e0d2943ddf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd25ef4d0d1b940c895dede7abebd1468', 'request_id': 'req-6d19363b-00fa-4d49-8da3-87e0d2943ddf'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:14:59.868157 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-6d19363b-00fa-4d49-8da3-87e0d2943ddf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd25ef4d0d1b940c895dede7abebd1468', 'request_id': 'req-6d19363b-00fa-4d49-8da3-87e0d2943ddf'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:14:59.892909 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-6d19363b-00fa-4d49-8da3-87e0d2943ddf admin admin] The token cache is being invalidate because project d25ef4d0d1b940c895dede7abebd1468 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80298) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 12:14:59.893932 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-6d19363b-00fa-4d49-8da3-87e0d2943ddf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:14:59.894845 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-6d19363b-00fa-4d49-8da3-87e0d2943ddf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:14:59.903040 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5355f196-2456-4dc4-875f-105a8bda9f13 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:14:59.903686 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5355f196-2456-4dc4-875f-105a8bda9f13 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:14:59.904112 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5355f196-2456-4dc4-875f-105a8bda9f13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:14:59.915366 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 872/1732] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Jul 25 12:14:59 2022] DELETE /identity/v3/projects/d25ef4d0d1b940c895dede7abebd1468 => generated 0 bytes in 530 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 12:14:59.956120 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-5355f196-2456-4dc4-875f-105a8bda9f13 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:14:59.957621 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-5355f196-2456-4dc4-875f-105a8bda9f13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:15:00.104856 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 861/1733] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:14:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:15:00.115526 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39753d1d-cce6-47ba-95d9-5d0b8bf16452 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:00.115526 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39753d1d-cce6-47ba-95d9-5d0b8bf16452 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:00.115526 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39753d1d-cce6-47ba-95d9-5d0b8bf16452 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:00.148605 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:15:00.165713 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-39753d1d-cce6-47ba-95d9-5d0b8bf16452 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:15:00.180782 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-39753d1d-cce6-47ba-95d9-5d0b8bf16452 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:15:00.206779 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5355f196-2456-4dc4-875f-105a8bda9f13 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:00.207939 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 873/1734] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:15:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:15:00.226391 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5355f196-2456-4dc4-875f-105a8bda9f13 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:00.246904 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d9780a-8979-4b30-b4b0-076b6f3988f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:00.247686 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d9780a-8979-4b30-b4b0-076b6f3988f6 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:00.247973 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d9780a-8979-4b30-b4b0-076b6f3988f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:00.248251 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d9780a-8979-4b30-b4b0-076b6f3988f6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:00.249233 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d9780a-8979-4b30-b4b0-076b6f3988f6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:15:00.249347 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d9780a-8979-4b30-b4b0-076b6f3988f6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:00.250897 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d9780a-8979-4b30-b4b0-076b6f3988f6 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:00.279920 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 862/1735] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Jul 25 12:15:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:15:00.286532 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39753d1d-cce6-47ba-95d9-5d0b8bf16452 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:00.304337 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39753d1d-cce6-47ba-95d9-5d0b8bf16452 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:00.311656 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725427dd-fbf9-488e-ad67-2ee1c4569edd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:00.312884 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725427dd-fbf9-488e-ad67-2ee1c4569edd admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:00.313027 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725427dd-fbf9-488e-ad67-2ee1c4569edd admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:00.313469 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725427dd-fbf9-488e-ad67-2ee1c4569edd admin admin] PATH_INFO: `/v3/projects` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:00.314298 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725427dd-fbf9-488e-ad67-2ee1c4569edd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:00.316910 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725427dd-fbf9-488e-ad67-2ee1c4569edd admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:00.361142 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 874/1736] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Jul 25 12:15:00 2022] POST /identity/v3/projects => generated 369 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:15:00.361476 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d9780a-8979-4b30-b4b0-076b6f3988f6 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:00.377649 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d9780a-8979-4b30-b4b0-076b6f3988f6 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:00.515743 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a306f9c-f844-4440-8066-9f6fd0993cca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:00.516578 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a306f9c-f844-4440-8066-9f6fd0993cca admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:00.516964 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a306f9c-f844-4440-8066-9f6fd0993cca admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:00.517439 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a306f9c-f844-4440-8066-9f6fd0993cca admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:00.519765 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a306f9c-f844-4440-8066-9f6fd0993cca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:00.526380 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a306f9c-f844-4440-8066-9f6fd0993cca admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:00.654395 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725427dd-fbf9-488e-ad67-2ee1c4569edd admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:00.660543 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 863/1737] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Jul 25 12:15:00 2022] POST /identity/v3/users => generated 285 bytes in 294 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:15:00.685826 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725427dd-fbf9-488e-ad67-2ee1c4569edd admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:00.700597 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1763b6ae-058a-4ca7-b4e0-1b20030f9526 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:00.701651 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1763b6ae-058a-4ca7-b4e0-1b20030f9526 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:00.702095 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1763b6ae-058a-4ca7-b4e0-1b20030f9526 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:00.702514 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1763b6ae-058a-4ca7-b4e0-1b20030f9526 admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:00.703524 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1763b6ae-058a-4ca7-b4e0-1b20030f9526 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:00.706397 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1763b6ae-058a-4ca7-b4e0-1b20030f9526 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:00.730565 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a306f9c-f844-4440-8066-9f6fd0993cca admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:00.741294 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 875/1738] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Jul 25 12:15:00 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:15:00.746074 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a306f9c-f844-4440-8066-9f6fd0993cca admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:00.776796 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b119ae9f-92af-499d-9af0-13fc543acf1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:00.780797 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b119ae9f-92af-499d-9af0-13fc543acf1d admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:00.787368 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b119ae9f-92af-499d-9af0-13fc543acf1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:00.787913 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b119ae9f-92af-499d-9af0-13fc543acf1d admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:00.789183 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b119ae9f-92af-499d-9af0-13fc543acf1d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:00.802741 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b119ae9f-92af-499d-9af0-13fc543acf1d admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:00.868145 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 864/1739] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Jul 25 12:15:00 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:15:00.869605 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1763b6ae-058a-4ca7-b4e0-1b20030f9526 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:00.901096 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1763b6ae-058a-4ca7-b4e0-1b20030f9526 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:00.916590 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fbcc2b-d937-49a0-b1db-0ab479b30694 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:00.917902 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fbcc2b-d937-49a0-b1db-0ab479b30694 admin admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:00.918368 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fbcc2b-d937-49a0-b1db-0ab479b30694 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:00.918810 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fbcc2b-d937-49a0-b1db-0ab479b30694 admin admin] PATH_INFO: `/v3/projects/c0789bb5a39847e7ab305bc3c595f349/users/0d3c1ff996014fc1a30b1973c0d8f277/roles/472313ae69174578824df0bed6390744` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:00.978765 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fbcc2b-d937-49a0-b1db-0ab479b30694 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c0789bb5a39847e7ab305bc3c595f349, user_id=0d3c1ff996014fc1a30b1973c0d8f277, role_id=472313ae69174578824df0bed6390744)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:00.982605 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fbcc2b-d937-49a0-b1db-0ab479b30694 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:01.059438 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 876/1740] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Jul 25 12:15:00 2022] PUT /identity/v3/projects/c0789bb5a39847e7ab305bc3c595f349/users/0d3c1ff996014fc1a30b1973c0d8f277/roles/472313ae69174578824df0bed6390744 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:15:01.061870 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9cb023-def0-4e89-a315-60258ef7f0f2 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:01.062228 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9cb023-def0-4e89-a315-60258ef7f0f2 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:01.062497 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9cb023-def0-4e89-a315-60258ef7f0f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:01.103947 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03197af6-eb37-4b83-9f5b-77700a2960b2 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:01.104534 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03197af6-eb37-4b83-9f5b-77700a2960b2 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:01.105153 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03197af6-eb37-4b83-9f5b-77700a2960b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:01.157633 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-03197af6-eb37-4b83-9f5b-77700a2960b2 None None] MFA Rules not processed for user `c5b98bbcc2d640de837d141f83dfe924`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:15:01.162651 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-03197af6-eb37-4b83-9f5b-77700a2960b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:15:01.163170 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-2e9cb023-def0-4e89-a315-60258ef7f0f2 None None] MFA Rules not processed for user `0d3c1ff996014fc1a30b1973c0d8f277`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:15:01.171039 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-2e9cb023-def0-4e89-a315-60258ef7f0f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:15:01.308221 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:15:01.473721 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 877/1741] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:15:01 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 372 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:15:01.491881 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 865/1742] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:15:01 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 431 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 12:15:01.497762 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03197af6-eb37-4b83-9f5b-77700a2960b2 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:01.668203 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03197af6-eb37-4b83-9f5b-77700a2960b2 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:01.674219 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb683420-3519-4900-8272-d7e288adad4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:01.692101 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb683420-3519-4900-8272-d7e288adad4f admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:01.692391 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb683420-3519-4900-8272-d7e288adad4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:01.692688 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb683420-3519-4900-8272-d7e288adad4f admin admin] PATH_INFO: `/v3/projects` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:01.694245 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb683420-3519-4900-8272-d7e288adad4f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:01.708579 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb683420-3519-4900-8272-d7e288adad4f admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:01.794662 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 878/1743] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Jul 25 12:15:01 2022] POST /identity/v3/projects => generated 369 bytes in 298 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:15:01.796737 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9cb023-def0-4e89-a315-60258ef7f0f2 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:01.826458 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9cb023-def0-4e89-a315-60258ef7f0f2 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:02.132872 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa1cfce-a32d-405d-8664-9b1030143df1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:02.134225 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa1cfce-a32d-405d-8664-9b1030143df1 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:02.134517 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa1cfce-a32d-405d-8664-9b1030143df1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:02.134802 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa1cfce-a32d-405d-8664-9b1030143df1 admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:02.135518 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa1cfce-a32d-405d-8664-9b1030143df1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:02.153249 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa1cfce-a32d-405d-8664-9b1030143df1 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:02.235914 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 866/1744] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Jul 25 12:15:01 2022] POST /identity/v3/users => generated 291 bytes in 438 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:15:02.238158 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb683420-3519-4900-8272-d7e288adad4f admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:02.278226 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb683420-3519-4900-8272-d7e288adad4f admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:02.322017 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f57234-f535-4749-9fbd-defb7d3d2ac0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:02.325714 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f57234-f535-4749-9fbd-defb7d3d2ac0 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:02.328026 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f57234-f535-4749-9fbd-defb7d3d2ac0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:02.330524 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f57234-f535-4749-9fbd-defb7d3d2ac0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:02.336740 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f57234-f535-4749-9fbd-defb7d3d2ac0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:02.341489 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f57234-f535-4749-9fbd-defb7d3d2ac0 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:02.422213 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 879/1745] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Jul 25 12:15:02 2022] GET /identity/v3/roles => generated 1420 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:15:02.423603 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa1cfce-a32d-405d-8664-9b1030143df1 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:02.445319 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa1cfce-a32d-405d-8664-9b1030143df1 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:02.467350 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac11abe2-00b2-4846-bba4-fb281a20bc5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:02.469051 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac11abe2-00b2-4846-bba4-fb281a20bc5e admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:02.469549 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac11abe2-00b2-4846-bba4-fb281a20bc5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:02.471602 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac11abe2-00b2-4846-bba4-fb281a20bc5e admin admin] PATH_INFO: `/v3/projects/357e0a6ae0534d3eac8d1e7650eeb03f/users/65b44c8e6dd74251ade240829d307a79/roles/a7d20885b6d94928a34b3cc5b3a00006` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:02.512198 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac11abe2-00b2-4846-bba4-fb281a20bc5e admin admin] RBAC: Authorizing `identity:create_grant(project_id=357e0a6ae0534d3eac8d1e7650eeb03f, user_id=65b44c8e6dd74251ade240829d307a79, role_id=a7d20885b6d94928a34b3cc5b3a00006)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:02.521301 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac11abe2-00b2-4846-bba4-fb281a20bc5e admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:02.571670 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 867/1746] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Jul 25 12:15:02 2022] PUT /identity/v3/projects/357e0a6ae0534d3eac8d1e7650eeb03f/users/65b44c8e6dd74251ade240829d307a79/roles/a7d20885b6d94928a34b3cc5b3a00006 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:15:02.576385 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f57234-f535-4749-9fbd-defb7d3d2ac0 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:02.614235 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f57234-f535-4749-9fbd-defb7d3d2ac0 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:02.696406 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:15:02.711225 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac11abe2-00b2-4846-bba4-fb281a20bc5e admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:02.713848 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-ac11abe2-00b2-4846-bba4-fb281a20bc5e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:15:02.813975 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfd2617-24c1-48d2-b864-26c23fd91973 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:02.814828 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfd2617-24c1-48d2-b864-26c23fd91973 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:02.815356 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfd2617-24c1-48d2-b864-26c23fd91973 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:02.815860 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfd2617-24c1-48d2-b864-26c23fd91973 admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:02.831453 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfd2617-24c1-48d2-b864-26c23fd91973 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:02.833189 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfd2617-24c1-48d2-b864-26c23fd91973 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:02.878081 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 880/1747] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Jul 25 12:15:02 2022] GET /identity/v3/roles => generated 1420 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:15:02.881015 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfd2617-24c1-48d2-b864-26c23fd91973 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:02.898556 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac11abe2-00b2-4846-bba4-fb281a20bc5e admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:02.910050 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfd2617-24c1-48d2-b864-26c23fd91973 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:02.930357 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013294f3-3847-474d-a979-94c34d646867 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:02.931334 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013294f3-3847-474d-a979-94c34d646867 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:02.931664 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013294f3-3847-474d-a979-94c34d646867 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:02.932761 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013294f3-3847-474d-a979-94c34d646867 admin admin] PATH_INFO: `/v3/users/921e5815a8044de4a08927ddf18371dc` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:02.937117 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013294f3-3847-474d-a979-94c34d646867 admin admin] RBAC: Authorizing `identity:delete_user(user_id=921e5815a8044de4a08927ddf18371dc)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:02.944147 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7d1bc5-c0ea-4167-9134-1e136b22e53e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:02.948365 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013294f3-3847-474d-a979-94c34d646867 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:02.952191 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7d1bc5-c0ea-4167-9134-1e136b22e53e admin admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:02.952487 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7d1bc5-c0ea-4167-9134-1e136b22e53e admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:02.952780 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7d1bc5-c0ea-4167-9134-1e136b22e53e admin admin] PATH_INFO: `/v3/projects/357e0a6ae0534d3eac8d1e7650eeb03f/users/65b44c8e6dd74251ade240829d307a79/roles/a7d20885b6d94928a34b3cc5b3a00006` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:02.966704 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7d1bc5-c0ea-4167-9134-1e136b22e53e admin admin] RBAC: Authorizing `identity:create_grant(project_id=357e0a6ae0534d3eac8d1e7650eeb03f, user_id=65b44c8e6dd74251ade240829d307a79, role_id=a7d20885b6d94928a34b3cc5b3a00006)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:02.994115 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7d1bc5-c0ea-4167-9134-1e136b22e53e admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:03.026481 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 881/1748] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Jul 25 12:15:02 2022] PUT /identity/v3/projects/357e0a6ae0534d3eac8d1e7650eeb03f/users/65b44c8e6dd74251ade240829d307a79/roles/a7d20885b6d94928a34b3cc5b3a00006 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:15:03.038358 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186b4cb2-b163-41d1-968d-e714e2115b66 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:03.038920 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186b4cb2-b163-41d1-968d-e714e2115b66 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:03.040107 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186b4cb2-b163-41d1-968d-e714e2115b66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:03.120521 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-186b4cb2-b163-41d1-968d-e714e2115b66 None None] MFA Rules not processed for user `65b44c8e6dd74251ade240829d307a79`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:15:03.123060 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-186b4cb2-b163-41d1-968d-e714e2115b66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:15:03.142658 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=90,keystone:INSERT=4,keystone:DELETE=6 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:15:03.164086 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-013294f3-3847-474d-a979-94c34d646867 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '921e5815a8044de4a08927ddf18371dc'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:03.218195 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-013294f3-3847-474d-a979-94c34d646867 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '921e5815a8044de4a08927ddf18371dc'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:03.237667 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-013294f3-3847-474d-a979-94c34d646867 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '921e5815a8044de4a08927ddf18371dc'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:03.263201 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-013294f3-3847-474d-a979-94c34d646867 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '921e5815a8044de4a08927ddf18371dc'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:03.264947 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-013294f3-3847-474d-a979-94c34d646867 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '921e5815a8044de4a08927ddf18371dc'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:03.269380 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 882/1749] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:15:03 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:15:03.277911 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 868/1750] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Jul 25 12:15:02 2022] DELETE /identity/v3/users/921e5815a8044de4a08927ddf18371dc => generated 0 bytes in 567 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:15:03.284651 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc51603d-d6d1-46ee-8e65-3318c5eb2f16 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:03.285112 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc51603d-d6d1-46ee-8e65-3318c5eb2f16 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:03.285663 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc51603d-d6d1-46ee-8e65-3318c5eb2f16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:03.292732 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013294f3-3847-474d-a979-94c34d646867 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:03.294191 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-013294f3-3847-474d-a979-94c34d646867 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:15:03.331630 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-dc51603d-d6d1-46ee-8e65-3318c5eb2f16 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:15:03.334143 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-dc51603d-d6d1-46ee-8e65-3318c5eb2f16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:15:03.573715 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013294f3-3847-474d-a979-94c34d646867 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:03.592410 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40c6b61-040f-4356-add2-8ed8fad78cf6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f89ce6f0cf04373a414d927fa0f8a7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:03.594370 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40c6b61-040f-4356-add2-8ed8fad78cf6 service cinder] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:03.598046 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40c6b61-040f-4356-add2-8ed8fad78cf6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:03.598531 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40c6b61-040f-4356-add2-8ed8fad78cf6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:03.603829 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-a40c6b61-040f-4356-add2-8ed8fad78cf6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:15:03.634353 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 883/1751] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:15:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 351 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 12:15:03.638251 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcc179a-a12f-4e58-9667-2c5697c0de44 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:03.638561 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcc179a-a12f-4e58-9667-2c5697c0de44 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:03.638850 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcc179a-a12f-4e58-9667-2c5697c0de44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:03.788155 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40c6b61-040f-4356-add2-8ed8fad78cf6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:03.789707 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40c6b61-040f-4356-add2-8ed8fad78cf6 service cinder] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:03.824376 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 869/1752] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:15:03 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 532 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 12:15:03.827230 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40c6b61-040f-4356-add2-8ed8fad78cf6 service cinder] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:03.833727 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-a40c6b61-040f-4356-add2-8ed8fad78cf6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:15:03.851109 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:15:03.859405 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-7bcc179a-a12f-4e58-9667-2c5697c0de44 None None] MFA Rules not processed for user `93207b858d294d049fa56d45810a40aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:15:03.862870 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-7bcc179a-a12f-4e58-9667-2c5697c0de44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:15:04.058771 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40c6b61-040f-4356-add2-8ed8fad78cf6 service cinder] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:04.081094 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b826840d-2ccd-4893-a6ee-77e2e9e464df service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51733781201243e8abf9eca39b207750', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:04.081803 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b826840d-2ccd-4893-a6ee-77e2e9e464df service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:04.082126 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b826840d-2ccd-4893-a6ee-77e2e9e464df service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:04.082405 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b826840d-2ccd-4893-a6ee-77e2e9e464df service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:04.161169 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 884/1753] 10.222.0.42 () {58 vars in 1009 bytes} [Mon Jul 25 12:15:03 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 526 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:15:04.164894 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b826840d-2ccd-4893-a6ee-77e2e9e464df service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:04.176271 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b826840d-2ccd-4893-a6ee-77e2e9e464df service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:04.212696 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 870/1754] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:15:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:15:04.488599 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcc179a-a12f-4e58-9667-2c5697c0de44 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:04.490562 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-7bcc179a-a12f-4e58-9667-2c5697c0de44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:15:04.707197 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcc179a-a12f-4e58-9667-2c5697c0de44 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:04.729901 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da585054-641f-471e-82be-93ccf753c195 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bd782ea66364eeb96592096eae63d4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:04.732533 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da585054-641f-471e-82be-93ccf753c195 service swift] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:04.735069 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da585054-641f-471e-82be-93ccf753c195 service swift] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:04.735680 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da585054-641f-471e-82be-93ccf753c195 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:04.766946 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da585054-641f-471e-82be-93ccf753c195 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:04.771663 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da585054-641f-471e-82be-93ccf753c195 service swift] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:04.784450 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 885/1755] 10.222.0.42 () {60 vars in 1384 bytes} [Mon Jul 25 12:15:04 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 25 12:15:04.909864 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b826840d-2ccd-4893-a6ee-77e2e9e464df service neutron] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:04.911798 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-b826840d-2ccd-4893-a6ee-77e2e9e464df service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:15:04.960635 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b826840d-2ccd-4893-a6ee-77e2e9e464df service neutron] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:04.978329 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81be17dc-830e-4365-bfc0-08399c612094 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:04.979840 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81be17dc-830e-4365-bfc0-08399c612094 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:04.982181 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81be17dc-830e-4365-bfc0-08399c612094 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:04.983084 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81be17dc-830e-4365-bfc0-08399c612094 admin admin] PATH_INFO: `/v3/users/be1a55307e1240478db8ee40a53f4560` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:04.987333 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81be17dc-830e-4365-bfc0-08399c612094 admin admin] RBAC: Authorizing `identity:delete_user(user_id=be1a55307e1240478db8ee40a53f4560)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:04.993646 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81be17dc-830e-4365-bfc0-08399c612094 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:05.004323 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:15:05.132575 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-81be17dc-830e-4365-bfc0-08399c612094 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'be1a55307e1240478db8ee40a53f4560'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:05.178511 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-81be17dc-830e-4365-bfc0-08399c612094 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'be1a55307e1240478db8ee40a53f4560'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:05.208057 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-81be17dc-830e-4365-bfc0-08399c612094 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'be1a55307e1240478db8ee40a53f4560'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:05.270058 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-81be17dc-830e-4365-bfc0-08399c612094 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'be1a55307e1240478db8ee40a53f4560'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:05.277826 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-81be17dc-830e-4365-bfc0-08399c612094 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'be1a55307e1240478db8ee40a53f4560'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:05.307289 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 871/1756] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Jul 25 12:15:04 2022] DELETE /identity/v3/users/be1a55307e1240478db8ee40a53f4560 => generated 0 bytes in 399 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:15:05.311847 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afccc497-d95c-46b6-9280-eff2dea995f4 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:05.312342 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afccc497-d95c-46b6-9280-eff2dea995f4 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:05.312472 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afccc497-d95c-46b6-9280-eff2dea995f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:05.377380 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-afccc497-d95c-46b6-9280-eff2dea995f4 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:15:05.378648 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-afccc497-d95c-46b6-9280-eff2dea995f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:15:05.711762 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 886/1757] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:15:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 396 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:15:06.010675 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81be17dc-830e-4365-bfc0-08399c612094 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:06.012160 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-81be17dc-830e-4365-bfc0-08399c612094 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:15:06.360902 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81be17dc-830e-4365-bfc0-08399c612094 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:06.371187 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1803eaac-c890-4c5b-b569-338ee6cf42eb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51733781201243e8abf9eca39b207750', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:06.371343 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1803eaac-c890-4c5b-b569-338ee6cf42eb service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:06.371658 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1803eaac-c890-4c5b-b569-338ee6cf42eb service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:06.371941 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1803eaac-c890-4c5b-b569-338ee6cf42eb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:06.409890 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1803eaac-c890-4c5b-b569-338ee6cf42eb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:06.415701 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1803eaac-c890-4c5b-b569-338ee6cf42eb service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:06.459245 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 872/1758] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:15:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 448 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 12:15:07.469955 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afccc497-d95c-46b6-9280-eff2dea995f4 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:07.472334 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-afccc497-d95c-46b6-9280-eff2dea995f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:15:07.511522 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afccc497-d95c-46b6-9280-eff2dea995f4 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:07.544063 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f932b6-1183-436b-bd2b-fc5b08b81a83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:07.554508 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f932b6-1183-436b-bd2b-fc5b08b81a83 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:07.555186 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f932b6-1183-436b-bd2b-fc5b08b81a83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:07.556210 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f932b6-1183-436b-bd2b-fc5b08b81a83 admin admin] PATH_INFO: `/v3/users/0d3c1ff996014fc1a30b1973c0d8f277` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:07.559480 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f932b6-1183-436b-bd2b-fc5b08b81a83 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0d3c1ff996014fc1a30b1973c0d8f277)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:07.571524 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f932b6-1183-436b-bd2b-fc5b08b81a83 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:07.606353 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:15:07.667340 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0e0210-c093-4a78-bea9-54d44572ae61 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:07.667867 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0e0210-c093-4a78-bea9-54d44572ae61 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:07.667949 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0e0210-c093-4a78-bea9-54d44572ae61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:07.725456 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-5c0e0210-c093-4a78-bea9-54d44572ae61 None None] MFA Rules not processed for user `3848eae5b8df48eb9816ba0d7485f7c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:15:07.735224 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-5c0e0210-c093-4a78-bea9-54d44572ae61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:15:07.741449 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-62f932b6-1183-436b-bd2b-fc5b08b81a83 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0d3c1ff996014fc1a30b1973c0d8f277'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:07.745698 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-62f932b6-1183-436b-bd2b-fc5b08b81a83 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0d3c1ff996014fc1a30b1973c0d8f277'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:07.774097 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-62f932b6-1183-436b-bd2b-fc5b08b81a83 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0d3c1ff996014fc1a30b1973c0d8f277'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:07.829114 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-62f932b6-1183-436b-bd2b-fc5b08b81a83 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0d3c1ff996014fc1a30b1973c0d8f277'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:07.859913 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-62f932b6-1183-436b-bd2b-fc5b08b81a83 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0d3c1ff996014fc1a30b1973c0d8f277'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:07.881399 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 887/1759] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Jul 25 12:15:07 2022] DELETE /identity/v3/users/0d3c1ff996014fc1a30b1973c0d8f277 => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:15:07.886279 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e70810-8fec-449f-a60f-ffa37b24d785 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:07.886708 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e70810-8fec-449f-a60f-ffa37b24d785 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:07.887108 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e70810-8fec-449f-a60f-ffa37b24d785 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:07.911341 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 873/1760] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:15:07 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:15:07.952116 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-d8e70810-8fec-449f-a60f-ffa37b24d785 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:15:07.953568 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-d8e70810-8fec-449f-a60f-ffa37b24d785 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:15:08.053693 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 888/1761] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:15:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 12:15:08.121186 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0e0210-c093-4a78-bea9-54d44572ae61 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:08.122644 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-5c0e0210-c093-4a78-bea9-54d44572ae61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:15:08.141076 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0e0210-c093-4a78-bea9-54d44572ae61 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:08.224348 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:15:08.305739 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96003fd3-a5f3-4423-bc32-7e92e4fc07ac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51733781201243e8abf9eca39b207750', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:08.306385 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96003fd3-a5f3-4423-bc32-7e92e4fc07ac service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:08.306680 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96003fd3-a5f3-4423-bc32-7e92e4fc07ac service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:08.315875 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96003fd3-a5f3-4423-bc32-7e92e4fc07ac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:08.343237 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96003fd3-a5f3-4423-bc32-7e92e4fc07ac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:08.346395 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96003fd3-a5f3-4423-bc32-7e92e4fc07ac service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:08.399151 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 874/1762] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:15:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 278 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:15:08.600205 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abaa2bf-ade6-4cf8-a5ab-a3f1664d4ee0 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:08.601081 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abaa2bf-ade6-4cf8-a5ab-a3f1664d4ee0 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:08.602126 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abaa2bf-ade6-4cf8-a5ab-a3f1664d4ee0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:08.670937 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-7abaa2bf-ade6-4cf8-a5ab-a3f1664d4ee0 None None] MFA Rules not processed for user `51733781201243e8abf9eca39b207750`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:15:08.679976 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-7abaa2bf-ade6-4cf8-a5ab-a3f1664d4ee0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:15:08.715627 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 889/1763] 10.222.0.39 () {58 vars in 1015 bytes} [Mon Jul 25 12:15:08 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:15:09.104128 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96003fd3-a5f3-4423-bc32-7e92e4fc07ac service neutron] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:09.114597 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96003fd3-a5f3-4423-bc32-7e92e4fc07ac service neutron] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:09.131381 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d32788-1c0b-4a02-bd23-9111e72ab683 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51733781201243e8abf9eca39b207750', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:09.132558 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d32788-1c0b-4a02-bd23-9111e72ab683 service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:09.135840 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d32788-1c0b-4a02-bd23-9111e72ab683 service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:09.138818 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d32788-1c0b-4a02-bd23-9111e72ab683 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:09.169548 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d32788-1c0b-4a02-bd23-9111e72ab683 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:09.173386 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d32788-1c0b-4a02-bd23-9111e72ab683 service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:09.204403 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 875/1764] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:15:09 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:15:10.167788 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abaa2bf-ade6-4cf8-a5ab-a3f1664d4ee0 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:10.170485 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-7abaa2bf-ade6-4cf8-a5ab-a3f1664d4ee0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:15:10.188878 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d32788-1c0b-4a02-bd23-9111e72ab683 service neutron] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:10.191350 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-11d32788-1c0b-4a02-bd23-9111e72ab683 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:15:10.216420 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abaa2bf-ade6-4cf8-a5ab-a3f1664d4ee0 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:10.233540 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d32788-1c0b-4a02-bd23-9111e72ab683 service neutron] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:10.248907 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55124fdc-a3de-4245-87fa-b87407eff5ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:10.251073 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55124fdc-a3de-4245-87fa-b87407eff5ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:10.254244 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55124fdc-a3de-4245-87fa-b87407eff5ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:10.256595 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55124fdc-a3de-4245-87fa-b87407eff5ba admin admin] PATH_INFO: `/v3/projects/eac1f4d434044492a139f4d5bad957f5` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:10.266255 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55124fdc-a3de-4245-87fa-b87407eff5ba admin admin] RBAC: Authorizing `identity:delete_project(project_id=eac1f4d434044492a139f4d5bad957f5)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:10.267457 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b3cfe5-9f7b-4da0-a086-1d75fc737d14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:10.269353 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b3cfe5-9f7b-4da0-a086-1d75fc737d14 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:10.271094 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b3cfe5-9f7b-4da0-a086-1d75fc737d14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:10.271643 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b3cfe5-9f7b-4da0-a086-1d75fc737d14 admin admin] PATH_INFO: `/v3/projects/c0789bb5a39847e7ab305bc3c595f349` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:10.275786 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55124fdc-a3de-4245-87fa-b87407eff5ba admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:10.276650 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b3cfe5-9f7b-4da0-a086-1d75fc737d14 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c0789bb5a39847e7ab305bc3c595f349)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:10.279009 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b3cfe5-9f7b-4da0-a086-1d75fc737d14 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:10.513346 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-01b3cfe5-9f7b-4da0-a086-1d75fc737d14 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c0789bb5a39847e7ab305bc3c595f349', 'request_id': 'req-01b3cfe5-9f7b-4da0-a086-1d75fc737d14'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:10.516320 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-55124fdc-a3de-4245-87fa-b87407eff5ba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eac1f4d434044492a139f4d5bad957f5', 'request_id': 'req-55124fdc-a3de-4245-87fa-b87407eff5ba'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:10.534109 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-01b3cfe5-9f7b-4da0-a086-1d75fc737d14 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c0789bb5a39847e7ab305bc3c595f349', 'request_id': 'req-01b3cfe5-9f7b-4da0-a086-1d75fc737d14'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:10.540447 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-55124fdc-a3de-4245-87fa-b87407eff5ba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eac1f4d434044492a139f4d5bad957f5', 'request_id': 'req-55124fdc-a3de-4245-87fa-b87407eff5ba'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:10.553712 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-01b3cfe5-9f7b-4da0-a086-1d75fc737d14 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c0789bb5a39847e7ab305bc3c595f349', 'request_id': 'req-01b3cfe5-9f7b-4da0-a086-1d75fc737d14'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:10.556868 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-55124fdc-a3de-4245-87fa-b87407eff5ba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eac1f4d434044492a139f4d5bad957f5', 'request_id': 'req-55124fdc-a3de-4245-87fa-b87407eff5ba'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:10.595012 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-01b3cfe5-9f7b-4da0-a086-1d75fc737d14 admin admin] The token cache is being invalidate because project c0789bb5a39847e7ab305bc3c595f349 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 12:15:10.595401 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-01b3cfe5-9f7b-4da0-a086-1d75fc737d14 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:10.596747 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-55124fdc-a3de-4245-87fa-b87407eff5ba admin admin] The token cache is being invalidate because project eac1f4d434044492a139f4d5bad957f5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80298) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 12:15:10.599825 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-55124fdc-a3de-4245-87fa-b87407eff5ba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:10.600899 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-55124fdc-a3de-4245-87fa-b87407eff5ba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:10.601666 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-01b3cfe5-9f7b-4da0-a086-1d75fc737d14 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:10.603600 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 890/1765] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Jul 25 12:15:10 2022] DELETE /identity/v3/projects/eac1f4d434044492a139f4d5bad957f5 => generated 0 bytes in 436 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:15:10.607162 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55124fdc-a3de-4245-87fa-b87407eff5ba admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:10.607579 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 876/1766] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Jul 25 12:15:10 2022] DELETE /identity/v3/projects/c0789bb5a39847e7ab305bc3c595f349 => generated 0 bytes in 421 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 12:15:10.610737 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-55124fdc-a3de-4245-87fa-b87407eff5ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:15:10.611295 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b3cfe5-9f7b-4da0-a086-1d75fc737d14 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:10.614569 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-01b3cfe5-9f7b-4da0-a086-1d75fc737d14 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:15:10.621148 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:15:10.825158 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b3cfe5-9f7b-4da0-a086-1d75fc737d14 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:10.893025 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55124fdc-a3de-4245-87fa-b87407eff5ba admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:10.913230 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e943d1-beb4-452a-a480-13d152fa93c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:10.915177 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e943d1-beb4-452a-a480-13d152fa93c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:10.917017 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e943d1-beb4-452a-a480-13d152fa93c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:10.918123 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e943d1-beb4-452a-a480-13d152fa93c7 admin admin] PATH_INFO: `/v3/users/c6887c60d1cd4a91a68c976d6f1fdafa` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:10.921144 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e943d1-beb4-452a-a480-13d152fa93c7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c6887c60d1cd4a91a68c976d6f1fdafa)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:10.923237 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e943d1-beb4-452a-a480-13d152fa93c7 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:11.021807 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:15:11.053726 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb612db-d5d6-4a16-83f8-26f874d3cd1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:11.066977 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-15e943d1-beb4-452a-a480-13d152fa93c7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c6887c60d1cd4a91a68c976d6f1fdafa'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:11.068062 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-15e943d1-beb4-452a-a480-13d152fa93c7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c6887c60d1cd4a91a68c976d6f1fdafa'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:11.069373 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb612db-d5d6-4a16-83f8-26f874d3cd1d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:11.069676 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb612db-d5d6-4a16-83f8-26f874d3cd1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:11.069989 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb612db-d5d6-4a16-83f8-26f874d3cd1d admin admin] PATH_INFO: `/v3/users/65b44c8e6dd74251ade240829d307a79` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:11.081514 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb612db-d5d6-4a16-83f8-26f874d3cd1d admin admin] RBAC: Authorizing `identity:delete_user(user_id=65b44c8e6dd74251ade240829d307a79)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:11.093417 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb612db-d5d6-4a16-83f8-26f874d3cd1d admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:11.107011 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-15e943d1-beb4-452a-a480-13d152fa93c7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c6887c60d1cd4a91a68c976d6f1fdafa'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:11.156492 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-15e943d1-beb4-452a-a480-13d152fa93c7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c6887c60d1cd4a91a68c976d6f1fdafa'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:11.171233 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-15e943d1-beb4-452a-a480-13d152fa93c7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c6887c60d1cd4a91a68c976d6f1fdafa'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:11.188976 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-dcb612db-d5d6-4a16-83f8-26f874d3cd1d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '65b44c8e6dd74251ade240829d307a79'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:11.190030 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 891/1767] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Jul 25 12:15:10 2022] DELETE /identity/v3/users/c6887c60d1cd4a91a68c976d6f1fdafa => generated 0 bytes in 583 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 12:15:11.191102 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e943d1-beb4-452a-a480-13d152fa93c7 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:11.200960 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-15e943d1-beb4-452a-a480-13d152fa93c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:15:11.212836 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-dcb612db-d5d6-4a16-83f8-26f874d3cd1d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '65b44c8e6dd74251ade240829d307a79'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:11.228796 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-dcb612db-d5d6-4a16-83f8-26f874d3cd1d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '65b44c8e6dd74251ade240829d307a79'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:11.274787 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-dcb612db-d5d6-4a16-83f8-26f874d3cd1d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '65b44c8e6dd74251ade240829d307a79'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:11.278941 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-dcb612db-d5d6-4a16-83f8-26f874d3cd1d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '65b44c8e6dd74251ade240829d307a79'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:11.300148 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 877/1768] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Jul 25 12:15:10 2022] DELETE /identity/v3/users/65b44c8e6dd74251ade240829d307a79 => generated 0 bytes in 689 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:15:11.446489 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e943d1-beb4-452a-a480-13d152fa93c7 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:11.602507 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:15:11.700145 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e1eb6b-81d1-46cf-8c5d-a90b59d64373 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:11.701036 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e1eb6b-81d1-46cf-8c5d-a90b59d64373 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:11.701333 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e1eb6b-81d1-46cf-8c5d-a90b59d64373 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:11.701619 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e1eb6b-81d1-46cf-8c5d-a90b59d64373 admin admin] PATH_INFO: `/v3/projects/6d8cb4a828cd49619c5f85a8e8881bc5` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:11.707989 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e1eb6b-81d1-46cf-8c5d-a90b59d64373 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6d8cb4a828cd49619c5f85a8e8881bc5)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:11.710404 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e1eb6b-81d1-46cf-8c5d-a90b59d64373 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:11.835135 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-e5e1eb6b-81d1-46cf-8c5d-a90b59d64373 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6d8cb4a828cd49619c5f85a8e8881bc5', 'request_id': 'req-e5e1eb6b-81d1-46cf-8c5d-a90b59d64373'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:11.862540 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-e5e1eb6b-81d1-46cf-8c5d-a90b59d64373 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6d8cb4a828cd49619c5f85a8e8881bc5', 'request_id': 'req-e5e1eb6b-81d1-46cf-8c5d-a90b59d64373'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:11.889617 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-e5e1eb6b-81d1-46cf-8c5d-a90b59d64373 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6d8cb4a828cd49619c5f85a8e8881bc5', 'request_id': 'req-e5e1eb6b-81d1-46cf-8c5d-a90b59d64373'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:11.934414 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-e5e1eb6b-81d1-46cf-8c5d-a90b59d64373 admin admin] The token cache is being invalidate because project 6d8cb4a828cd49619c5f85a8e8881bc5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80298) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 12:15:11.936183 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-e5e1eb6b-81d1-46cf-8c5d-a90b59d64373 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:11.938188 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-e5e1eb6b-81d1-46cf-8c5d-a90b59d64373 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:11.946816 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 892/1769] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Jul 25 12:15:11 2022] DELETE /identity/v3/projects/6d8cb4a828cd49619c5f85a8e8881bc5 => generated 0 bytes in 756 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:15:11.951066 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb612db-d5d6-4a16-83f8-26f874d3cd1d admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:11.952671 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-dcb612db-d5d6-4a16-83f8-26f874d3cd1d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:15:11.985532 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb612db-d5d6-4a16-83f8-26f874d3cd1d admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:12.240993 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c206e167-9768-4b86-acbe-b24cafa1e3a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:12.246888 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c206e167-9768-4b86-acbe-b24cafa1e3a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:12.248941 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c206e167-9768-4b86-acbe-b24cafa1e3a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:12.250065 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c206e167-9768-4b86-acbe-b24cafa1e3a9 admin admin] PATH_INFO: `/v3/users/885dac3726694b288b0738d68d98223c` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:12.254068 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c206e167-9768-4b86-acbe-b24cafa1e3a9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=885dac3726694b288b0738d68d98223c)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:12.260016 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c206e167-9768-4b86-acbe-b24cafa1e3a9 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:12.443372 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-c206e167-9768-4b86-acbe-b24cafa1e3a9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '885dac3726694b288b0738d68d98223c'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:12.475255 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-c206e167-9768-4b86-acbe-b24cafa1e3a9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '885dac3726694b288b0738d68d98223c'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:12.490113 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-c206e167-9768-4b86-acbe-b24cafa1e3a9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '885dac3726694b288b0738d68d98223c'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:12.519110 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:15:12.536563 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-c206e167-9768-4b86-acbe-b24cafa1e3a9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '885dac3726694b288b0738d68d98223c'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:12.537700 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-c206e167-9768-4b86-acbe-b24cafa1e3a9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '885dac3726694b288b0738d68d98223c'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:12.557102 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 878/1770] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Jul 25 12:15:11 2022] DELETE /identity/v3/users/885dac3726694b288b0738d68d98223c => generated 0 bytes in 607 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:15:13.797151 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e1eb6b-81d1-46cf-8c5d-a90b59d64373 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:13.798809 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-e5e1eb6b-81d1-46cf-8c5d-a90b59d64373 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:15:13.824575 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e1eb6b-81d1-46cf-8c5d-a90b59d64373 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:13.975930 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3a943c-a89b-47d6-b116-2be4641a3633 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:13.976885 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3a943c-a89b-47d6-b116-2be4641a3633 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:13.977171 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3a943c-a89b-47d6-b116-2be4641a3633 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:13.977452 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3a943c-a89b-47d6-b116-2be4641a3633 admin admin] PATH_INFO: `/v3/projects/3a49657f567c4de5a4a546ddd7e587ef` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:13.990449 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3a943c-a89b-47d6-b116-2be4641a3633 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3a49657f567c4de5a4a546ddd7e587ef)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:13.997370 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3a943c-a89b-47d6-b116-2be4641a3633 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:14.155242 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-8e3a943c-a89b-47d6-b116-2be4641a3633 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3a49657f567c4de5a4a546ddd7e587ef', 'request_id': 'req-8e3a943c-a89b-47d6-b116-2be4641a3633'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:14.167768 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-8e3a943c-a89b-47d6-b116-2be4641a3633 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3a49657f567c4de5a4a546ddd7e587ef', 'request_id': 'req-8e3a943c-a89b-47d6-b116-2be4641a3633'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:14.186574 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:15:14.189667 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-8e3a943c-a89b-47d6-b116-2be4641a3633 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3a49657f567c4de5a4a546ddd7e587ef', 'request_id': 'req-8e3a943c-a89b-47d6-b116-2be4641a3633'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:14.213327 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-8e3a943c-a89b-47d6-b116-2be4641a3633 admin admin] The token cache is being invalidate because project 3a49657f567c4de5a4a546ddd7e587ef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80298) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 12:15:14.213327 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-8e3a943c-a89b-47d6-b116-2be4641a3633 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:14.214121 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-8e3a943c-a89b-47d6-b116-2be4641a3633 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:14.216280 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 893/1771] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Jul 25 12:15:13 2022] DELETE /identity/v3/projects/3a49657f567c4de5a4a546ddd7e587ef => generated 0 bytes in 419 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:15:14.221263 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48666f6c-bf58-413f-a921-b067a5f3b7ac None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:14.221823 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48666f6c-bf58-413f-a921-b067a5f3b7ac None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:14.221823 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48666f6c-bf58-413f-a921-b067a5f3b7ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:14.260935 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-48666f6c-bf58-413f-a921-b067a5f3b7ac None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:15:14.262296 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-48666f6c-bf58-413f-a921-b067a5f3b7ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:15:14.322198 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ef587c-addd-4256-b413-9bda948d3c18 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:14.335455 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ef587c-addd-4256-b413-9bda948d3c18 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:14.335455 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ef587c-addd-4256-b413-9bda948d3c18 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:14.335455 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 894/1772] 10.222.0.42 () {54 vars in 921 bytes} [Mon Jul 25 12:15:14 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 12:15:14.381216 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80424334-f2e8-4ddf-aa93-2344dc3fa718 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:14.381704 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80424334-f2e8-4ddf-aa93-2344dc3fa718 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:14.382183 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80424334-f2e8-4ddf-aa93-2344dc3fa718 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:14.391476 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 879/1773] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:15:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:15:14.408140 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05047b2-133b-46ba-b9fe-1e36cc8e5e00 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:14.408437 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05047b2-133b-46ba-b9fe-1e36cc8e5e00 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:14.408728 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05047b2-133b-46ba-b9fe-1e36cc8e5e00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:14.419679 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-80424334-f2e8-4ddf-aa93-2344dc3fa718 None None] MFA Rules not processed for user `f67142a1193d47c7bd33d7040268233c`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:15:14.421302 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-80424334-f2e8-4ddf-aa93-2344dc3fa718 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:15:14.477808 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-a05047b2-133b-46ba-b9fe-1e36cc8e5e00 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:15:14.479654 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-a05047b2-133b-46ba-b9fe-1e36cc8e5e00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:15:14.500572 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 880/1774] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:15:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:15:14.506276 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05047b2-133b-46ba-b9fe-1e36cc8e5e00 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:14.530436 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05047b2-133b-46ba-b9fe-1e36cc8e5e00 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:14.550960 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b67197a-686c-4b9b-aa8e-1b78bb84b718 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:14.551757 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b67197a-686c-4b9b-aa8e-1b78bb84b718 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:14.552053 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b67197a-686c-4b9b-aa8e-1b78bb84b718 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:14.552349 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b67197a-686c-4b9b-aa8e-1b78bb84b718 admin admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:14.556609 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b67197a-686c-4b9b-aa8e-1b78bb84b718 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:15:14.557029 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b67197a-686c-4b9b-aa8e-1b78bb84b718 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:14.558921 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b67197a-686c-4b9b-aa8e-1b78bb84b718 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:14.580392 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 895/1775] 10.222.0.42 () {58 vars in 1018 bytes} [Mon Jul 25 12:15:14 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:15:14.595433 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80424334-f2e8-4ddf-aa93-2344dc3fa718 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:14.595911 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 881/1776] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Jul 25 12:15:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 12:15:14.629495 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80424334-f2e8-4ddf-aa93-2344dc3fa718 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:14.642444 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f02589-baf8-4f55-bf2d-ab8a4493a931 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:14.644033 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f02589-baf8-4f55-bf2d-ab8a4493a931 admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:14.647025 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f02589-baf8-4f55-bf2d-ab8a4493a931 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:14.648107 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f02589-baf8-4f55-bf2d-ab8a4493a931 admin admin] PATH_INFO: `/v3/projects` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:14.649464 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f02589-baf8-4f55-bf2d-ab8a4493a931 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:14.652199 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f02589-baf8-4f55-bf2d-ab8a4493a931 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:14.749934 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 896/1777] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Jul 25 12:15:14 2022] POST /identity/v3/projects => generated 387 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:15:14.754447 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b67197a-686c-4b9b-aa8e-1b78bb84b718 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:14.783863 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b67197a-686c-4b9b-aa8e-1b78bb84b718 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:14.951325 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212df3d1-37c1-431b-9e8e-12cd88f50a24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:14.952387 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212df3d1-37c1-431b-9e8e-12cd88f50a24 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:14.952830 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212df3d1-37c1-431b-9e8e-12cd88f50a24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:14.953265 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212df3d1-37c1-431b-9e8e-12cd88f50a24 admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:14.954114 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212df3d1-37c1-431b-9e8e-12cd88f50a24 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:14.960202 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212df3d1-37c1-431b-9e8e-12cd88f50a24 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:15.016109 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 882/1778] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Jul 25 12:15:14 2022] POST /identity/v3/users => generated 294 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:15:15.017104 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f02589-baf8-4f55-bf2d-ab8a4493a931 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:15.040309 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f02589-baf8-4f55-bf2d-ab8a4493a931 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:15.084374 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02083253-2404-4b94-bdd1-494bdf7a6c82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:15.085241 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02083253-2404-4b94-bdd1-494bdf7a6c82 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:15.086428 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02083253-2404-4b94-bdd1-494bdf7a6c82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:15.086823 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02083253-2404-4b94-bdd1-494bdf7a6c82 admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:15.087720 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02083253-2404-4b94-bdd1-494bdf7a6c82 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:15.093214 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02083253-2404-4b94-bdd1-494bdf7a6c82 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:15.141773 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 897/1779] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Jul 25 12:15:15 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 12:15:15.143429 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212df3d1-37c1-431b-9e8e-12cd88f50a24 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:15.156204 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212df3d1-37c1-431b-9e8e-12cd88f50a24 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:15.169828 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d8b669-9bfc-4ff2-a721-ab2f85baa5b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:15.170773 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d8b669-9bfc-4ff2-a721-ab2f85baa5b9 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:15.171001 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d8b669-9bfc-4ff2-a721-ab2f85baa5b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:15.171381 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d8b669-9bfc-4ff2-a721-ab2f85baa5b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:15.173017 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d8b669-9bfc-4ff2-a721-ab2f85baa5b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:15.188467 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d8b669-9bfc-4ff2-a721-ab2f85baa5b9 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:15.258362 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 883/1780] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Jul 25 12:15:15 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:15:15.262546 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02083253-2404-4b94-bdd1-494bdf7a6c82 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:15.290262 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02083253-2404-4b94-bdd1-494bdf7a6c82 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:15.322564 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a60493-82d6-428c-9cca-8828d6ef1a86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:15.326853 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a60493-82d6-428c-9cca-8828d6ef1a86 admin admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:15.327151 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a60493-82d6-428c-9cca-8828d6ef1a86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:15.327581 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a60493-82d6-428c-9cca-8828d6ef1a86 admin admin] PATH_INFO: `/v3/projects/d0ecb7d347874891b39b6519dd76d907/users/4c4161f950274f50893d17f13778556c/roles/472313ae69174578824df0bed6390744` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:15.390486 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a60493-82d6-428c-9cca-8828d6ef1a86 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d0ecb7d347874891b39b6519dd76d907, user_id=4c4161f950274f50893d17f13778556c, role_id=472313ae69174578824df0bed6390744)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:15.393411 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a60493-82d6-428c-9cca-8828d6ef1a86 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:15.446552 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 898/1781] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Jul 25 12:15:15 2022] PUT /identity/v3/projects/d0ecb7d347874891b39b6519dd76d907/users/4c4161f950274f50893d17f13778556c/roles/472313ae69174578824df0bed6390744 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:15:15.457048 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7234aeb-c81d-41af-90a4-a0c2b068745f None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:15.457470 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7234aeb-c81d-41af-90a4-a0c2b068745f None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:15.457805 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7234aeb-c81d-41af-90a4-a0c2b068745f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:15.545981 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-b7234aeb-c81d-41af-90a4-a0c2b068745f None None] MFA Rules not processed for user `4c4161f950274f50893d17f13778556c`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:15:15.547229 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-b7234aeb-c81d-41af-90a4-a0c2b068745f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:15:15.648773 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:15:15.671809 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a60493-82d6-428c-9cca-8828d6ef1a86 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:15.674549 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-75a60493-82d6-428c-9cca-8828d6ef1a86 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:15:15.756680 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 884/1782] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:15:15 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 300 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:15:15.758331 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7234aeb-c81d-41af-90a4-a0c2b068745f None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:15.980230 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a60493-82d6-428c-9cca-8828d6ef1a86 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:16.001888 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafc2f66-667a-456a-a7a9-1177dc1c4da6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:16.006880 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafc2f66-667a-456a-a7a9-1177dc1c4da6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:16.008187 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafc2f66-667a-456a-a7a9-1177dc1c4da6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:16.015876 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafc2f66-667a-456a-a7a9-1177dc1c4da6 admin admin] PATH_INFO: `/v3/projects/c2b8c9f9bfeb40baa4237d96504bd4dd` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:16.017744 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafc2f66-667a-456a-a7a9-1177dc1c4da6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c2b8c9f9bfeb40baa4237d96504bd4dd)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:16.026424 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafc2f66-667a-456a-a7a9-1177dc1c4da6 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:16.075104 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7234aeb-c81d-41af-90a4-a0c2b068745f None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:16.097497 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed373f0e-96dc-40df-8964-05a24c1c17db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:16.100900 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed373f0e-96dc-40df-8964-05a24c1c17db admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:16.101380 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed373f0e-96dc-40df-8964-05a24c1c17db admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:16.101786 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed373f0e-96dc-40df-8964-05a24c1c17db admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:16.104399 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed373f0e-96dc-40df-8964-05a24c1c17db admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:16.108262 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed373f0e-96dc-40df-8964-05a24c1c17db admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:16.151589 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:DELETE=2,keystone:SELECT=96,keystone:INSERT=2 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:15:16.215472 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 885/1783] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Jul 25 12:15:15 2022] POST /identity/v3/projects => generated 387 bytes in 458 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:15:16.216755 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed373f0e-96dc-40df-8964-05a24c1c17db admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:16.220201 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-fafc2f66-667a-456a-a7a9-1177dc1c4da6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c2b8c9f9bfeb40baa4237d96504bd4dd', 'request_id': 'req-fafc2f66-667a-456a-a7a9-1177dc1c4da6'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:16.225257 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-ed373f0e-96dc-40df-8964-05a24c1c17db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:15:16.237526 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-fafc2f66-667a-456a-a7a9-1177dc1c4da6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c2b8c9f9bfeb40baa4237d96504bd4dd', 'request_id': 'req-fafc2f66-667a-456a-a7a9-1177dc1c4da6'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:16.285866 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-fafc2f66-667a-456a-a7a9-1177dc1c4da6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c2b8c9f9bfeb40baa4237d96504bd4dd', 'request_id': 'req-fafc2f66-667a-456a-a7a9-1177dc1c4da6'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:16.357172 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-fafc2f66-667a-456a-a7a9-1177dc1c4da6 admin admin] The token cache is being invalidate because project c2b8c9f9bfeb40baa4237d96504bd4dd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80298) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 12:15:16.357479 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-fafc2f66-667a-456a-a7a9-1177dc1c4da6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:16.368254 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-fafc2f66-667a-456a-a7a9-1177dc1c4da6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:16.373697 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 899/1784] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Jul 25 12:15:15 2022] DELETE /identity/v3/projects/c2b8c9f9bfeb40baa4237d96504bd4dd => generated 0 bytes in 703 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:15:16.381856 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafc2f66-667a-456a-a7a9-1177dc1c4da6 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:16.389291 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-fafc2f66-667a-456a-a7a9-1177dc1c4da6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:15:16.568794 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed373f0e-96dc-40df-8964-05a24c1c17db admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:16.606325 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e003bea-ef21-4131-87a2-1474be366e71 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f67142a1193d47c7bd33d7040268233c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:16.609004 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e003bea-ef21-4131-87a2-1474be366e71 service nova] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:16.609456 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e003bea-ef21-4131-87a2-1474be366e71 service nova] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:16.610321 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e003bea-ef21-4131-87a2-1474be366e71 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:16.614214 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-5e003bea-ef21-4131-87a2-1474be366e71 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:15:16.648655 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafc2f66-667a-456a-a7a9-1177dc1c4da6 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:16.654843 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e003bea-ef21-4131-87a2-1474be366e71 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:16.662292 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e003bea-ef21-4131-87a2-1474be366e71 service nova] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:16.687077 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 886/1785] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:15:16 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 471 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:15:16.688286 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e003bea-ef21-4131-87a2-1474be366e71 service nova] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:16.690315 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-5e003bea-ef21-4131-87a2-1474be366e71 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:15:16.817258 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10376c13-0f43-46ac-8ed2-f9bc3110196b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:16.818625 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10376c13-0f43-46ac-8ed2-f9bc3110196b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:16.819481 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10376c13-0f43-46ac-8ed2-f9bc3110196b admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:16.819481 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10376c13-0f43-46ac-8ed2-f9bc3110196b admin admin] PATH_INFO: `/v3/projects/357e0a6ae0534d3eac8d1e7650eeb03f` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:16.822299 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10376c13-0f43-46ac-8ed2-f9bc3110196b admin admin] RBAC: Authorizing `identity:delete_project(project_id=357e0a6ae0534d3eac8d1e7650eeb03f)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:16.827745 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10376c13-0f43-46ac-8ed2-f9bc3110196b admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:16.841937 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:15:16.896940 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e003bea-ef21-4131-87a2-1474be366e71 service nova] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:16.907455 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42de63ce-2b93-4649-8f09-b976c6c60468 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:16.908535 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42de63ce-2b93-4649-8f09-b976c6c60468 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:16.909035 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42de63ce-2b93-4649-8f09-b976c6c60468 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:16.909511 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42de63ce-2b93-4649-8f09-b976c6c60468 admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:16.910731 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42de63ce-2b93-4649-8f09-b976c6c60468 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:16.920284 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42de63ce-2b93-4649-8f09-b976c6c60468 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:16.961443 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-10376c13-0f43-46ac-8ed2-f9bc3110196b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '357e0a6ae0534d3eac8d1e7650eeb03f', 'request_id': 'req-10376c13-0f43-46ac-8ed2-f9bc3110196b'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:16.972824 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 887/1786] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Jul 25 12:15:16 2022] POST /identity/v3/users => generated 294 bytes in 285 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:15:16.974764 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4319a9b8-a2b4-486c-a4f1-784284bad212 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:16.975201 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4319a9b8-a2b4-486c-a4f1-784284bad212 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:16.975744 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4319a9b8-a2b4-486c-a4f1-784284bad212 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:16.979111 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-10376c13-0f43-46ac-8ed2-f9bc3110196b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '357e0a6ae0534d3eac8d1e7650eeb03f', 'request_id': 'req-10376c13-0f43-46ac-8ed2-f9bc3110196b'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:16.987896 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:15:16.990039 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-10376c13-0f43-46ac-8ed2-f9bc3110196b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '357e0a6ae0534d3eac8d1e7650eeb03f', 'request_id': 'req-10376c13-0f43-46ac-8ed2-f9bc3110196b'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:17.016926 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-10376c13-0f43-46ac-8ed2-f9bc3110196b admin admin] The token cache is being invalidate because project 357e0a6ae0534d3eac8d1e7650eeb03f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80298) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 12:15:17.016926 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-10376c13-0f43-46ac-8ed2-f9bc3110196b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:17.019437 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-10376c13-0f43-46ac-8ed2-f9bc3110196b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:17.021717 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 900/1787] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Jul 25 12:15:16 2022] DELETE /identity/v3/projects/357e0a6ae0534d3eac8d1e7650eeb03f => generated 0 bytes in 647 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:15:17.024092 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10376c13-0f43-46ac-8ed2-f9bc3110196b admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:17.027518 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-10376c13-0f43-46ac-8ed2-f9bc3110196b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:15:17.029680 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-4319a9b8-a2b4-486c-a4f1-784284bad212 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:15:17.033532 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-4319a9b8-a2b4-486c-a4f1-784284bad212 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:15:17.187757 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 888/1788] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:15:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:15:17.187757 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545db5d2-d8c4-40c6-8c1a-53b5a0cdf255 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:17.187757 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545db5d2-d8c4-40c6-8c1a-53b5a0cdf255 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:17.187757 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545db5d2-d8c4-40c6-8c1a-53b5a0cdf255 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:17.194425 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10376c13-0f43-46ac-8ed2-f9bc3110196b admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:17.204633 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f8df7a-37d2-4361-a88b-73cd673f9bee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:17.206888 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f8df7a-37d2-4361-a88b-73cd673f9bee admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:17.207897 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f8df7a-37d2-4361-a88b-73cd673f9bee admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:17.209376 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f8df7a-37d2-4361-a88b-73cd673f9bee admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:17.211611 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f8df7a-37d2-4361-a88b-73cd673f9bee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:17.215904 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f8df7a-37d2-4361-a88b-73cd673f9bee admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:17.236945 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-545db5d2-d8c4-40c6-8c1a-53b5a0cdf255 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:15:17.240287 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-545db5d2-d8c4-40c6-8c1a-53b5a0cdf255 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:15:17.243730 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 901/1789] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Jul 25 12:15:17 2022] GET /identity/v3/roles => generated 1420 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:15:17.247136 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cc4c16-71cc-481e-819e-1ec6ee7b6ffa None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:17.247765 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cc4c16-71cc-481e-819e-1ec6ee7b6ffa None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:17.249040 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cc4c16-71cc-481e-819e-1ec6ee7b6ffa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:17.265866 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 889/1790] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:15:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:15:17.267451 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545db5d2-d8c4-40c6-8c1a-53b5a0cdf255 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:17.290179 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545db5d2-d8c4-40c6-8c1a-53b5a0cdf255 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:17.298684 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a8b40c-99e0-4d8a-880b-b0af48194588 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:17.300412 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a8b40c-99e0-4d8a-880b-b0af48194588 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:17.301260 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a8b40c-99e0-4d8a-880b-b0af48194588 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:17.302780 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a8b40c-99e0-4d8a-880b-b0af48194588 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:17.304299 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a8b40c-99e0-4d8a-880b-b0af48194588 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:17.307033 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a8b40c-99e0-4d8a-880b-b0af48194588 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:17.324468 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 890/1791] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Jul 25 12:15:17 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:15:17.326768 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50423491-b090-43a8-bfe7-178a45239bdf None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:17.327282 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50423491-b090-43a8-bfe7-178a45239bdf None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:17.328096 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50423491-b090-43a8-bfe7-178a45239bdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:17.329812 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-d7cc4c16-71cc-481e-819e-1ec6ee7b6ffa None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:15:17.331056 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-d7cc4c16-71cc-481e-819e-1ec6ee7b6ffa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:15:17.373115 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 902/1792] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:15:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:15:17.374198 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cc4c16-71cc-481e-819e-1ec6ee7b6ffa None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:17.384547 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-50423491-b090-43a8-bfe7-178a45239bdf None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:15:17.389419 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-50423491-b090-43a8-bfe7-178a45239bdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:15:17.417302 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cc4c16-71cc-481e-819e-1ec6ee7b6ffa None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:17.421195 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 891/1793] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:15:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:15:17.428336 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50423491-b090-43a8-bfe7-178a45239bdf None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:17.432265 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff9d65a-f718-42d4-9f34-b0c1bc426a3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:17.433187 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff9d65a-f718-42d4-9f34-b0c1bc426a3f admin admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:17.433476 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff9d65a-f718-42d4-9f34-b0c1bc426a3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:17.433826 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff9d65a-f718-42d4-9f34-b0c1bc426a3f admin admin] PATH_INFO: `/v3/projects/91029af1b4d24082992561293988d07c/users/bb97468743ab4d6cb3c476b7b521821d/roles/472313ae69174578824df0bed6390744` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:17.464392 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50423491-b090-43a8-bfe7-178a45239bdf None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:17.482634 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2aa54b-f6e7-4393-9255-32d66890ae95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:17.484342 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2aa54b-f6e7-4393-9255-32d66890ae95 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:17.484638 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2aa54b-f6e7-4393-9255-32d66890ae95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:17.486981 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2aa54b-f6e7-4393-9255-32d66890ae95 admin admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:17.489505 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2aa54b-f6e7-4393-9255-32d66890ae95 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:15:17.489826 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2aa54b-f6e7-4393-9255-32d66890ae95 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:17.497276 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff9d65a-f718-42d4-9f34-b0c1bc426a3f admin admin] RBAC: Authorizing `identity:create_grant(project_id=91029af1b4d24082992561293988d07c, user_id=bb97468743ab4d6cb3c476b7b521821d, role_id=472313ae69174578824df0bed6390744)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:17.498966 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2aa54b-f6e7-4393-9255-32d66890ae95 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:17.525855 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff9d65a-f718-42d4-9f34-b0c1bc426a3f admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:17.538308 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 892/1794] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Jul 25 12:15:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:15:17.539834 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2aa54b-f6e7-4393-9255-32d66890ae95 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:17.562462 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 903/1795] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Jul 25 12:15:17 2022] PUT /identity/v3/projects/91029af1b4d24082992561293988d07c/users/bb97468743ab4d6cb3c476b7b521821d/roles/472313ae69174578824df0bed6390744 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:15:17.564203 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2aa54b-f6e7-4393-9255-32d66890ae95 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:17.581889 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff9d65a-f718-42d4-9f34-b0c1bc426a3f admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:17.703967 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe072611-7f17-4cc3-be7c-855d7e7f603b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:17.704727 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe072611-7f17-4cc3-be7c-855d7e7f603b admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:17.705117 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe072611-7f17-4cc3-be7c-855d7e7f603b admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:17.705472 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe072611-7f17-4cc3-be7c-855d7e7f603b admin admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:17.707162 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe072611-7f17-4cc3-be7c-855d7e7f603b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:15:17.707586 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe072611-7f17-4cc3-be7c-855d7e7f603b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:17.709303 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe072611-7f17-4cc3-be7c-855d7e7f603b admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:17.742036 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 893/1796] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Jul 25 12:15:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:15:17.750641 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a914851d-3a4e-4389-85e5-27fbc49b5b9e None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:17.751857 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a914851d-3a4e-4389-85e5-27fbc49b5b9e None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:17.753696 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a914851d-3a4e-4389-85e5-27fbc49b5b9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:17.787859 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff9d65a-f718-42d4-9f34-b0c1bc426a3f admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:17.800142 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403743b4-f2bf-47d0-b97d-0ef01c028421 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:17.800839 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403743b4-f2bf-47d0-b97d-0ef01c028421 admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:17.801143 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403743b4-f2bf-47d0-b97d-0ef01c028421 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:17.801407 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403743b4-f2bf-47d0-b97d-0ef01c028421 admin admin] PATH_INFO: `/v3/projects` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:17.802271 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403743b4-f2bf-47d0-b97d-0ef01c028421 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:17.804892 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403743b4-f2bf-47d0-b97d-0ef01c028421 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:17.838733 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-a914851d-3a4e-4389-85e5-27fbc49b5b9e None None] MFA Rules not processed for user `bb97468743ab4d6cb3c476b7b521821d`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:15:17.840054 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-a914851d-3a4e-4389-85e5-27fbc49b5b9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:15:17.903940 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 904/1797] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Jul 25 12:15:17 2022] POST /identity/v3/projects => generated 387 bytes in 338 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:15:17.903940 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a64f77-2f39-4640-a605-0f3fadd62375 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:17.909157 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a64f77-2f39-4640-a605-0f3fadd62375 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:17.909499 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a64f77-2f39-4640-a605-0f3fadd62375 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:17.913138 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 905/1798] 10.222.0.42 () {54 vars in 921 bytes} [Mon Jul 25 12:15:17 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:15:17.928481 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:15:17.928848 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a64f77-2f39-4640-a605-0f3fadd62375 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:18.041942 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:15:18.088927 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 894/1799] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:15:17 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 342 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:15:18.088927 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a914851d-3a4e-4389-85e5-27fbc49b5b9e None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:18.090948 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a64f77-2f39-4640-a605-0f3fadd62375 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:18.101578 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028a4559-70a5-48b5-afa1-3df1a6d1d6f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:18.102229 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028a4559-70a5-48b5-afa1-3df1a6d1d6f0 admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:18.102519 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028a4559-70a5-48b5-afa1-3df1a6d1d6f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:18.102802 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028a4559-70a5-48b5-afa1-3df1a6d1d6f0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:18.103511 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028a4559-70a5-48b5-afa1-3df1a6d1d6f0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:18.106712 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028a4559-70a5-48b5-afa1-3df1a6d1d6f0 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:18.143441 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a914851d-3a4e-4389-85e5-27fbc49b5b9e None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:18.175139 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1901821-2228-4c2c-9495-3cf680a8b046 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:18.178766 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1901821-2228-4c2c-9495-3cf680a8b046 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:18.179080 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1901821-2228-4c2c-9495-3cf680a8b046 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:18.182476 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1901821-2228-4c2c-9495-3cf680a8b046 admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:18.183208 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1901821-2228-4c2c-9495-3cf680a8b046 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:18.185802 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 906/1800] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Jul 25 12:15:17 2022] POST /identity/v3/projects => generated 387 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:15:18.190495 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfeeb6f-6e82-4fd8-8fb0-600856347d94 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:18.190786 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfeeb6f-6e82-4fd8-8fb0-600856347d94 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:18.191067 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfeeb6f-6e82-4fd8-8fb0-600856347d94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:18.201551 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1901821-2228-4c2c-9495-3cf680a8b046 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:18.239411 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-bdfeeb6f-6e82-4fd8-8fb0-600856347d94 None None] MFA Rules not processed for user `f67142a1193d47c7bd33d7040268233c`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:15:18.242572 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-bdfeeb6f-6e82-4fd8-8fb0-600856347d94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:15:18.293418 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 895/1801] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Jul 25 12:15:18 2022] POST /identity/v3/users => generated 294 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:15:18.298490 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1901821-2228-4c2c-9495-3cf680a8b046 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:18.354535 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1901821-2228-4c2c-9495-3cf680a8b046 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:18.467639 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 907/1802] 10.222.0.42 () {58 vars in 1018 bytes} [Mon Jul 25 12:15:18 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 277 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:15:18.469749 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfeeb6f-6e82-4fd8-8fb0-600856347d94 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:18.648660 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec95b9bd-8272-4627-926f-e808aefc922d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:18.650585 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec95b9bd-8272-4627-926f-e808aefc922d admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:18.653874 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec95b9bd-8272-4627-926f-e808aefc922d admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:18.654276 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec95b9bd-8272-4627-926f-e808aefc922d admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:18.655770 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec95b9bd-8272-4627-926f-e808aefc922d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:18.660191 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec95b9bd-8272-4627-926f-e808aefc922d admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:18.729688 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfeeb6f-6e82-4fd8-8fb0-600856347d94 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:18.746096 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1ccdca-87fb-4ffc-8366-6200fe9166fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:18.746891 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1ccdca-87fb-4ffc-8366-6200fe9166fb admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:18.747426 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1ccdca-87fb-4ffc-8366-6200fe9166fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:18.747804 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1ccdca-87fb-4ffc-8366-6200fe9166fb admin admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:18.748729 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1ccdca-87fb-4ffc-8366-6200fe9166fb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:18.751388 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1ccdca-87fb-4ffc-8366-6200fe9166fb admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:18.817012 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 896/1803] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Jul 25 12:15:18 2022] POST /identity/v3/projects => generated 387 bytes in 523 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:15:18.819066 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec95b9bd-8272-4627-926f-e808aefc922d admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:18.839442 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 908/1804] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Jul 25 12:15:18 2022] POST /identity/v3/users => generated 294 bytes in 368 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:15:18.839442 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1ccdca-87fb-4ffc-8366-6200fe9166fb admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:18.873598 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1ccdca-87fb-4ffc-8366-6200fe9166fb admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:18.911014 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec95b9bd-8272-4627-926f-e808aefc922d admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:18.923274 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545a09ad-4906-4306-9c0e-309fa5d29085 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:18.924037 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545a09ad-4906-4306-9c0e-309fa5d29085 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:18.924324 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545a09ad-4906-4306-9c0e-309fa5d29085 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:18.924605 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545a09ad-4906-4306-9c0e-309fa5d29085 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:18.932360 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545a09ad-4906-4306-9c0e-309fa5d29085 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:18.938846 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545a09ad-4906-4306-9c0e-309fa5d29085 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:18.986615 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 897/1805] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Jul 25 12:15:18 2022] GET /identity/v3/roles => generated 1420 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:15:18.987498 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545a09ad-4906-4306-9c0e-309fa5d29085 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:19.014339 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:15:19.050862 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545a09ad-4906-4306-9c0e-309fa5d29085 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:19.062732 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f43fcd-3f8b-43b7-8b01-133a856f0546 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:19.063440 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f43fcd-3f8b-43b7-8b01-133a856f0546 admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:19.063795 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f43fcd-3f8b-43b7-8b01-133a856f0546 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:19.064035 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f43fcd-3f8b-43b7-8b01-133a856f0546 admin admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:19.070466 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79dc8ac1-dedd-4919-a37a-91f94a830907 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:19.071158 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dc8ac1-dedd-4919-a37a-91f94a830907 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:19.071447 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dc8ac1-dedd-4919-a37a-91f94a830907 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:19.071757 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dc8ac1-dedd-4919-a37a-91f94a830907 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:19.072505 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f43fcd-3f8b-43b7-8b01-133a856f0546 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:19.074757 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f43fcd-3f8b-43b7-8b01-133a856f0546 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:19.077881 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79dc8ac1-dedd-4919-a37a-91f94a830907 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:19.085487 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79dc8ac1-dedd-4919-a37a-91f94a830907 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:19.122788 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 898/1806] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Jul 25 12:15:18 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:15:19.123767 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79dc8ac1-dedd-4919-a37a-91f94a830907 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:19.152704 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 909/1807] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Jul 25 12:15:18 2022] POST /identity/v3/users => generated 300 bytes in 315 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:15:19.153604 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f43fcd-3f8b-43b7-8b01-133a856f0546 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:19.155536 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79dc8ac1-dedd-4919-a37a-91f94a830907 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:19.167760 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f43fcd-3f8b-43b7-8b01-133a856f0546 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:19.170090 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293d8d6a-850e-45d0-a4ef-63aa07642f65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:19.170772 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293d8d6a-850e-45d0-a4ef-63aa07642f65 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:19.171060 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293d8d6a-850e-45d0-a4ef-63aa07642f65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:19.171347 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293d8d6a-850e-45d0-a4ef-63aa07642f65 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:19.172112 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293d8d6a-850e-45d0-a4ef-63aa07642f65 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:19.173770 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293d8d6a-850e-45d0-a4ef-63aa07642f65 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:19.188427 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c225be0-ea14-41b7-8e24-73d408810032 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:19.189379 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c225be0-ea14-41b7-8e24-73d408810032 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:19.189379 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c225be0-ea14-41b7-8e24-73d408810032 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:19.189660 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c225be0-ea14-41b7-8e24-73d408810032 admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:19.191415 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c225be0-ea14-41b7-8e24-73d408810032 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:19.201511 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c225be0-ea14-41b7-8e24-73d408810032 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:19.205469 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 899/1808] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Jul 25 12:15:19 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:15:19.210228 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293d8d6a-850e-45d0-a4ef-63aa07642f65 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:19.246250 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293d8d6a-850e-45d0-a4ef-63aa07642f65 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:19.247235 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 910/1809] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Jul 25 12:15:19 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:15:19.248060 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c225be0-ea14-41b7-8e24-73d408810032 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:19.253538 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00d5988-7d7b-4003-8e16-0984bd5a31ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:19.254998 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00d5988-7d7b-4003-8e16-0984bd5a31ea admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:19.255290 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00d5988-7d7b-4003-8e16-0984bd5a31ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:19.255785 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00d5988-7d7b-4003-8e16-0984bd5a31ea admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:19.256693 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00d5988-7d7b-4003-8e16-0984bd5a31ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:19.259103 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00d5988-7d7b-4003-8e16-0984bd5a31ea admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:19.261017 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:15:19.285914 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 900/1810] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Jul 25 12:15:19 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:15:19.286635 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00d5988-7d7b-4003-8e16-0984bd5a31ea admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:19.294168 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c225be0-ea14-41b7-8e24-73d408810032 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:19.304230 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1414de02-2ea1-43ef-a6ea-68634dfd2f5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:19.313046 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1414de02-2ea1-43ef-a6ea-68634dfd2f5b admin admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:19.313168 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1414de02-2ea1-43ef-a6ea-68634dfd2f5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:19.313401 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1414de02-2ea1-43ef-a6ea-68634dfd2f5b admin admin] PATH_INFO: `/v3/projects/fc3f1b387a2f4119aa5adbed8181c26a/users/f3d3cf99393c49abbcb6e0f4b4301997/roles/472313ae69174578824df0bed6390744` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:19.313631 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00d5988-7d7b-4003-8e16-0984bd5a31ea admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:19.321791 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cf7686-f97a-4036-b36f-ad5aeb0e3ab9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:19.322760 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cf7686-f97a-4036-b36f-ad5aeb0e3ab9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:19.323053 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cf7686-f97a-4036-b36f-ad5aeb0e3ab9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:19.323704 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cf7686-f97a-4036-b36f-ad5aeb0e3ab9 admin admin] PATH_INFO: `/v3/projects/0ce09e226a444016b47e4e286061e66f/users/5d49c1f43f234bb89df11549d1a57d8e/roles/472313ae69174578824df0bed6390744` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:19.349372 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1414de02-2ea1-43ef-a6ea-68634dfd2f5b admin admin] RBAC: Authorizing `identity:create_grant(project_id=fc3f1b387a2f4119aa5adbed8181c26a, user_id=f3d3cf99393c49abbcb6e0f4b4301997, role_id=472313ae69174578824df0bed6390744)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:19.355128 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1414de02-2ea1-43ef-a6ea-68634dfd2f5b admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:19.357189 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cf7686-f97a-4036-b36f-ad5aeb0e3ab9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ce09e226a444016b47e4e286061e66f, user_id=5d49c1f43f234bb89df11549d1a57d8e, role_id=472313ae69174578824df0bed6390744)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:19.362259 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cf7686-f97a-4036-b36f-ad5aeb0e3ab9 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:19.397195 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 911/1812] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Jul 25 12:15:19 2022] PUT /identity/v3/projects/fc3f1b387a2f4119aa5adbed8181c26a/users/f3d3cf99393c49abbcb6e0f4b4301997/roles/472313ae69174578824df0bed6390744 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:15:19.398793 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1414de02-2ea1-43ef-a6ea-68634dfd2f5b admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:19.399651 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 901/1812] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Jul 25 12:15:19 2022] PUT /identity/v3/projects/0ce09e226a444016b47e4e286061e66f/users/5d49c1f43f234bb89df11549d1a57d8e/roles/472313ae69174578824df0bed6390744 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:15:19.401173 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7f74b0-9d2a-4464-ad54-a0d4fdae2a1a None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:19.401463 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7f74b0-9d2a-4464-ad54-a0d4fdae2a1a None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:19.401741 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7f74b0-9d2a-4464-ad54-a0d4fdae2a1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:19.450587 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-6c7f74b0-9d2a-4464-ad54-a0d4fdae2a1a None None] MFA Rules not processed for user `f3d3cf99393c49abbcb6e0f4b4301997`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:15:19.451939 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-6c7f74b0-9d2a-4464-ad54-a0d4fdae2a1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:15:19.515115 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1414de02-2ea1-43ef-a6ea-68634dfd2f5b admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:19.554037 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 902/1813] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:15:19 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:15:19.559324 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c738a5-b25a-42db-879a-a5dbe59de4c8 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:19.559688 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c738a5-b25a-42db-879a-a5dbe59de4c8 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:19.560125 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c738a5-b25a-42db-879a-a5dbe59de4c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:19.617526 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-68c738a5-b25a-42db-879a-a5dbe59de4c8 None None] MFA Rules not processed for user `5d49c1f43f234bb89df11549d1a57d8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:15:19.619501 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-68c738a5-b25a-42db-879a-a5dbe59de4c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:15:19.637706 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4beb0e3-264c-4fdd-bc31-3abaa5a0454d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:19.638648 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4beb0e3-264c-4fdd-bc31-3abaa5a0454d admin admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:19.638941 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4beb0e3-264c-4fdd-bc31-3abaa5a0454d admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:19.639230 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4beb0e3-264c-4fdd-bc31-3abaa5a0454d admin admin] PATH_INFO: `/v3/projects/767d37cf832442adb1838fe183f42b65/users/4d28296ffc8b44839d2b225db131a80c/roles/a7d20885b6d94928a34b3cc5b3a00006` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:19.677846 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4beb0e3-264c-4fdd-bc31-3abaa5a0454d admin admin] RBAC: Authorizing `identity:create_grant(project_id=767d37cf832442adb1838fe183f42b65, user_id=4d28296ffc8b44839d2b225db131a80c, role_id=a7d20885b6d94928a34b3cc5b3a00006)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:19.681588 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4beb0e3-264c-4fdd-bc31-3abaa5a0454d admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:19.713567 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 912/1814] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Jul 25 12:15:19 2022] PUT /identity/v3/projects/767d37cf832442adb1838fe183f42b65/users/4d28296ffc8b44839d2b225db131a80c/roles/a7d20885b6d94928a34b3cc5b3a00006 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:15:19.714629 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4beb0e3-264c-4fdd-bc31-3abaa5a0454d admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:19.716742 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-a4beb0e3-264c-4fdd-bc31-3abaa5a0454d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:15:19.763038 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:15:19.785817 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 903/1815] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:15:19 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:15:19.786740 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c738a5-b25a-42db-879a-a5dbe59de4c8 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:19.799028 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c738a5-b25a-42db-879a-a5dbe59de4c8 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:19.826500 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4beb0e3-264c-4fdd-bc31-3abaa5a0454d admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:19.833257 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815c6a29-72e6-42f5-8290-e229d577fdc6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f67142a1193d47c7bd33d7040268233c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:19.834093 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815c6a29-72e6-42f5-8290-e229d577fdc6 service nova] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:19.834849 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815c6a29-72e6-42f5-8290-e229d577fdc6 service nova] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:19.835245 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815c6a29-72e6-42f5-8290-e229d577fdc6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:19.848851 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815c6a29-72e6-42f5-8290-e229d577fdc6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:19.851401 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815c6a29-72e6-42f5-8290-e229d577fdc6 service nova] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:19.868006 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 913/1816] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:15:19 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:15:19.868824 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815c6a29-72e6-42f5-8290-e229d577fdc6 service nova] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:19.911792 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:15:19.936346 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a1f37d-58d4-4339-9bc4-74208ffcdd4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:19.937077 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a1f37d-58d4-4339-9bc4-74208ffcdd4d admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:19.937315 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a1f37d-58d4-4339-9bc4-74208ffcdd4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:19.937599 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a1f37d-58d4-4339-9bc4-74208ffcdd4d admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:19.938312 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a1f37d-58d4-4339-9bc4-74208ffcdd4d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:19.946472 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a1f37d-58d4-4339-9bc4-74208ffcdd4d admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:19.995761 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 904/1817] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Jul 25 12:15:19 2022] POST /identity/v3/projects => generated 387 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:15:19.995761 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a1f37d-58d4-4339-9bc4-74208ffcdd4d admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:19.997373 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815c6a29-72e6-42f5-8290-e229d577fdc6 service nova] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:20.078634 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29f33e2-a3c3-4726-a059-c0dbff37ecc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:20.079749 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29f33e2-a3c3-4726-a059-c0dbff37ecc8 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:20.080054 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29f33e2-a3c3-4726-a059-c0dbff37ecc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:20.080341 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29f33e2-a3c3-4726-a059-c0dbff37ecc8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:20.081583 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29f33e2-a3c3-4726-a059-c0dbff37ecc8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:20.083657 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29f33e2-a3c3-4726-a059-c0dbff37ecc8 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:20.096708 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a1f37d-58d4-4339-9bc4-74208ffcdd4d admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:20.098210 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 914/1818] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Jul 25 12:15:19 2022] GET /identity/v3/roles => generated 1420 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:15:20.099304 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29f33e2-a3c3-4726-a059-c0dbff37ecc8 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:20.105662 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc7fb2e-43b3-47a5-82ac-a3c563757f35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:20.106881 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc7fb2e-43b3-47a5-82ac-a3c563757f35 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:20.107301 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc7fb2e-43b3-47a5-82ac-a3c563757f35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:20.107750 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc7fb2e-43b3-47a5-82ac-a3c563757f35 admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:20.109217 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc7fb2e-43b3-47a5-82ac-a3c563757f35 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:20.111969 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc7fb2e-43b3-47a5-82ac-a3c563757f35 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:20.113629 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29f33e2-a3c3-4726-a059-c0dbff37ecc8 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:20.121676 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65fa36a-296a-4bc2-a1f2-c02d4c8926fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:20.122625 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65fa36a-296a-4bc2-a1f2-c02d4c8926fc admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:20.123064 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65fa36a-296a-4bc2-a1f2-c02d4c8926fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:20.123497 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65fa36a-296a-4bc2-a1f2-c02d4c8926fc admin admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:20.124497 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65fa36a-296a-4bc2-a1f2-c02d4c8926fc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:20.127910 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65fa36a-296a-4bc2-a1f2-c02d4c8926fc admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:20.155660 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 905/1819] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Jul 25 12:15:19 2022] POST /identity/v3/projects => generated 387 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:15:20.155660 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc7fb2e-43b3-47a5-82ac-a3c563757f35 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:20.161983 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 915/1820] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Jul 25 12:15:20 2022] POST /identity/v3/users => generated 300 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:15:20.162801 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65fa36a-296a-4bc2-a1f2-c02d4c8926fc admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:20.176125 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65fa36a-296a-4bc2-a1f2-c02d4c8926fc admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:20.250103 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:15:20.251583 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc7fb2e-43b3-47a5-82ac-a3c563757f35 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:20.257167 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c391ed-a969-4f62-81fe-3201107ef606 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:20.258592 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c391ed-a969-4f62-81fe-3201107ef606 admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:20.259453 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c391ed-a969-4f62-81fe-3201107ef606 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:20.259453 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c391ed-a969-4f62-81fe-3201107ef606 admin admin] PATH_INFO: `/v3/projects/767d37cf832442adb1838fe183f42b65/users/4d28296ffc8b44839d2b225db131a80c/roles/a7d20885b6d94928a34b3cc5b3a00006` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:20.262228 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c391ed-a969-4f62-81fe-3201107ef606 admin admin] RBAC: Authorizing `identity:create_grant(project_id=767d37cf832442adb1838fe183f42b65, user_id=4d28296ffc8b44839d2b225db131a80c, role_id=a7d20885b6d94928a34b3cc5b3a00006)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:20.265809 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c391ed-a969-4f62-81fe-3201107ef606 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:20.270688 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b675c176-9f5f-4d88-af11-414a04261311 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:20.271459 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b675c176-9f5f-4d88-af11-414a04261311 admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:20.271785 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b675c176-9f5f-4d88-af11-414a04261311 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:20.272117 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b675c176-9f5f-4d88-af11-414a04261311 admin admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:20.272901 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b675c176-9f5f-4d88-af11-414a04261311 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:20.275827 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b675c176-9f5f-4d88-af11-414a04261311 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:20.298140 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 906/1821] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Jul 25 12:15:20 2022] PUT /identity/v3/projects/767d37cf832442adb1838fe183f42b65/users/4d28296ffc8b44839d2b225db131a80c/roles/a7d20885b6d94928a34b3cc5b3a00006 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:15:20.298140 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c391ed-a969-4f62-81fe-3201107ef606 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:20.312653 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 916/1822] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Jul 25 12:15:20 2022] POST /identity/v3/users => generated 300 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:15:20.315062 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f760bb2e-5aa2-4b26-ad25-aed217567c85 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:20.315624 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f760bb2e-5aa2-4b26-ad25-aed217567c85 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:20.316885 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f760bb2e-5aa2-4b26-ad25-aed217567c85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:20.356349 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-f760bb2e-5aa2-4b26-ad25-aed217567c85 None None] MFA Rules not processed for user `4d28296ffc8b44839d2b225db131a80c`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:15:20.358442 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-f760bb2e-5aa2-4b26-ad25-aed217567c85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:15:20.386045 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:15:20.397385 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c391ed-a969-4f62-81fe-3201107ef606 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:20.407110 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045a834c-8034-4507-b88b-fef5f2c5840e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:20.409590 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045a834c-8034-4507-b88b-fef5f2c5840e admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:20.409888 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045a834c-8034-4507-b88b-fef5f2c5840e admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:20.410425 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045a834c-8034-4507-b88b-fef5f2c5840e admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:20.411202 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045a834c-8034-4507-b88b-fef5f2c5840e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:20.413031 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045a834c-8034-4507-b88b-fef5f2c5840e admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:20.432208 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 907/1823] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Jul 25 12:15:20 2022] GET /identity/v3/roles => generated 1420 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:15:20.433202 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045a834c-8034-4507-b88b-fef5f2c5840e admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:20.445851 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045a834c-8034-4507-b88b-fef5f2c5840e admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:20.451062 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1812a0-d566-4ab1-bc17-7f7d71d01cbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:20.451779 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1812a0-d566-4ab1-bc17-7f7d71d01cbc admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:20.452071 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1812a0-d566-4ab1-bc17-7f7d71d01cbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:20.452353 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1812a0-d566-4ab1-bc17-7f7d71d01cbc admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:20.453206 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1812a0-d566-4ab1-bc17-7f7d71d01cbc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:20.455409 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1812a0-d566-4ab1-bc17-7f7d71d01cbc admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:20.471384 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 908/1824] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Jul 25 12:15:20 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:15:20.472440 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1812a0-d566-4ab1-bc17-7f7d71d01cbc admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:20.484236 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1812a0-d566-4ab1-bc17-7f7d71d01cbc admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:20.489483 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66852117-a981-4d07-bdc9-70a882497c73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:20.490665 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66852117-a981-4d07-bdc9-70a882497c73 admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:20.490847 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66852117-a981-4d07-bdc9-70a882497c73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:20.492232 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66852117-a981-4d07-bdc9-70a882497c73 admin admin] PATH_INFO: `/v3/projects/8fa9b873c09d4eea9feaaf3973b6238f/users/d2e670c4ab554a0e82b219e6a99ad025/roles/a7d20885b6d94928a34b3cc5b3a00006` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:20.501594 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 917/1825] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:15:20 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:15:20.502554 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f760bb2e-5aa2-4b26-ad25-aed217567c85 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:20.518242 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66852117-a981-4d07-bdc9-70a882497c73 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8fa9b873c09d4eea9feaaf3973b6238f, user_id=d2e670c4ab554a0e82b219e6a99ad025, role_id=a7d20885b6d94928a34b3cc5b3a00006)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:20.523249 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66852117-a981-4d07-bdc9-70a882497c73 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:20.529864 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f760bb2e-5aa2-4b26-ad25-aed217567c85 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:20.534634 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002adce2-0add-4047-8d9a-cb3186990907 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:20.535927 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002adce2-0add-4047-8d9a-cb3186990907 admin admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:20.536130 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002adce2-0add-4047-8d9a-cb3186990907 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:20.536409 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002adce2-0add-4047-8d9a-cb3186990907 admin admin] PATH_INFO: `/v3/projects/3798ab22a9004d1ca57c0ddfe91991e7/users/8a9da76d97524aeca137c2ae26396f3f/roles/a7d20885b6d94928a34b3cc5b3a00006` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:20.559862 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 909/1826] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Jul 25 12:15:20 2022] PUT /identity/v3/projects/8fa9b873c09d4eea9feaaf3973b6238f/users/d2e670c4ab554a0e82b219e6a99ad025/roles/a7d20885b6d94928a34b3cc5b3a00006 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:15:20.561446 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66852117-a981-4d07-bdc9-70a882497c73 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:20.565365 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002adce2-0add-4047-8d9a-cb3186990907 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3798ab22a9004d1ca57c0ddfe91991e7, user_id=8a9da76d97524aeca137c2ae26396f3f, role_id=a7d20885b6d94928a34b3cc5b3a00006)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:20.565536 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-66852117-a981-4d07-bdc9-70a882497c73 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:15:20.568946 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002adce2-0add-4047-8d9a-cb3186990907 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:20.600242 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 918/1827] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Jul 25 12:15:20 2022] PUT /identity/v3/projects/3798ab22a9004d1ca57c0ddfe91991e7/users/8a9da76d97524aeca137c2ae26396f3f/roles/a7d20885b6d94928a34b3cc5b3a00006 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:15:20.602046 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002adce2-0add-4047-8d9a-cb3186990907 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:20.680410 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66852117-a981-4d07-bdc9-70a882497c73 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:20.700860 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002adce2-0add-4047-8d9a-cb3186990907 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:20.707903 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7952c9f-4c76-4cfc-a7aa-c1732bc425cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:20.710584 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7952c9f-4c76-4cfc-a7aa-c1732bc425cd admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:20.711256 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7952c9f-4c76-4cfc-a7aa-c1732bc425cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:20.711751 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7952c9f-4c76-4cfc-a7aa-c1732bc425cd admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:20.712717 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7952c9f-4c76-4cfc-a7aa-c1732bc425cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:20.715389 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7952c9f-4c76-4cfc-a7aa-c1732bc425cd admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:20.732866 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 919/1828] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Jul 25 12:15:20 2022] GET /identity/v3/roles => generated 1420 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:15:20.733745 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7952c9f-4c76-4cfc-a7aa-c1732bc425cd admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:20.757096 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7952c9f-4c76-4cfc-a7aa-c1732bc425cd admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:20.765512 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b51ff10-4007-433d-b49a-2fb423f8fbc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:20.766251 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b51ff10-4007-433d-b49a-2fb423f8fbc0 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:20.766591 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b51ff10-4007-433d-b49a-2fb423f8fbc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:20.767022 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b51ff10-4007-433d-b49a-2fb423f8fbc0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:20.767790 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b51ff10-4007-433d-b49a-2fb423f8fbc0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:20.773619 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b51ff10-4007-433d-b49a-2fb423f8fbc0 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:20.797189 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 920/1829] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Jul 25 12:15:20 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:15:20.798608 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b51ff10-4007-433d-b49a-2fb423f8fbc0 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:20.821698 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a518172a-977e-44e8-a195-90e50450eb99 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f89ce6f0cf04373a414d927fa0f8a7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:20.822483 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a518172a-977e-44e8-a195-90e50450eb99 service cinder] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:20.823929 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a518172a-977e-44e8-a195-90e50450eb99 service cinder] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:20.826044 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a518172a-977e-44e8-a195-90e50450eb99 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:20.836415 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b51ff10-4007-433d-b49a-2fb423f8fbc0 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:20.851956 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f39bc94-b2c1-44ab-b252-4387ba30c40e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:20.852994 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f39bc94-b2c1-44ab-b252-4387ba30c40e admin admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:20.853566 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f39bc94-b2c1-44ab-b252-4387ba30c40e admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:20.853784 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f39bc94-b2c1-44ab-b252-4387ba30c40e admin admin] PATH_INFO: `/v3/projects/8fa9b873c09d4eea9feaaf3973b6238f/users/d2e670c4ab554a0e82b219e6a99ad025/roles/a7d20885b6d94928a34b3cc5b3a00006` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:20.864814 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f39bc94-b2c1-44ab-b252-4387ba30c40e admin admin] RBAC: Authorizing `identity:create_grant(project_id=8fa9b873c09d4eea9feaaf3973b6238f, user_id=d2e670c4ab554a0e82b219e6a99ad025, role_id=a7d20885b6d94928a34b3cc5b3a00006)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:20.872734 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f39bc94-b2c1-44ab-b252-4387ba30c40e admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:20.905457 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 921/1830] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Jul 25 12:15:20 2022] PUT /identity/v3/projects/8fa9b873c09d4eea9feaaf3973b6238f/users/d2e670c4ab554a0e82b219e6a99ad025/roles/a7d20885b6d94928a34b3cc5b3a00006 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:15:20.906608 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f39bc94-b2c1-44ab-b252-4387ba30c40e admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:20.920772 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:15:20.945352 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a518172a-977e-44e8-a195-90e50450eb99 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:20.947041 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a518172a-977e-44e8-a195-90e50450eb99 service cinder] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:20.964476 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 910/1831] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:15:20 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:15:20.965832 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f219fc0-abe0-4d5c-83be-f96ed144f0e4 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:20.966198 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f219fc0-abe0-4d5c-83be-f96ed144f0e4 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:20.966553 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f219fc0-abe0-4d5c-83be-f96ed144f0e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:21.024375 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f39bc94-b2c1-44ab-b252-4387ba30c40e admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:21.033208 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-1f219fc0-abe0-4d5c-83be-f96ed144f0e4 None None] MFA Rules not processed for user `d2e670c4ab554a0e82b219e6a99ad025`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:15:21.035161 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-1f219fc0-abe0-4d5c-83be-f96ed144f0e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:15:21.046140 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b2d6e9-eadb-4d51-93b3-2442d108d12e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:21.047150 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b2d6e9-eadb-4d51-93b3-2442d108d12e admin admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:21.047516 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b2d6e9-eadb-4d51-93b3-2442d108d12e admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:21.047915 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b2d6e9-eadb-4d51-93b3-2442d108d12e admin admin] PATH_INFO: `/v3/projects/3798ab22a9004d1ca57c0ddfe91991e7/users/8a9da76d97524aeca137c2ae26396f3f/roles/a7d20885b6d94928a34b3cc5b3a00006` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:21.052091 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b2d6e9-eadb-4d51-93b3-2442d108d12e admin admin] RBAC: Authorizing `identity:create_grant(project_id=3798ab22a9004d1ca57c0ddfe91991e7, user_id=8a9da76d97524aeca137c2ae26396f3f, role_id=a7d20885b6d94928a34b3cc5b3a00006)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:21.055952 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b2d6e9-eadb-4d51-93b3-2442d108d12e admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:21.064877 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:15:21.069585 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 922/1832] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Jul 25 12:15:20 2022] PUT /identity/v3/projects/3798ab22a9004d1ca57c0ddfe91991e7/users/8a9da76d97524aeca137c2ae26396f3f/roles/a7d20885b6d94928a34b3cc5b3a00006 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:15:21.079879 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25152c6f-fe08-44e2-a9dd-385bb1c8f18b None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:21.080495 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25152c6f-fe08-44e2-a9dd-385bb1c8f18b None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:21.081086 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25152c6f-fe08-44e2-a9dd-385bb1c8f18b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:21.159649 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-25152c6f-fe08-44e2-a9dd-385bb1c8f18b None None] MFA Rules not processed for user `8a9da76d97524aeca137c2ae26396f3f`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:15:21.161217 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-25152c6f-fe08-44e2-a9dd-385bb1c8f18b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:15:21.166001 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 911/1833] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:15:20 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:15:21.177156 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f219fc0-abe0-4d5c-83be-f96ed144f0e4 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:21.179562 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-1f219fc0-abe0-4d5c-83be-f96ed144f0e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:15:21.273607 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f219fc0-abe0-4d5c-83be-f96ed144f0e4 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:21.279456 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 923/1834] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:15:21 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:15:21.321163 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25152c6f-fe08-44e2-a9dd-385bb1c8f18b None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:21.372051 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9c7004-d038-4ca5-be75-e545657a09b5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f89ce6f0cf04373a414d927fa0f8a7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:21.372795 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9c7004-d038-4ca5-be75-e545657a09b5 service cinder] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:21.373091 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9c7004-d038-4ca5-be75-e545657a09b5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:21.373378 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9c7004-d038-4ca5-be75-e545657a09b5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:21.425450 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25152c6f-fe08-44e2-a9dd-385bb1c8f18b None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:21.436645 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a52f59-d370-489e-9a2b-2552c3653d32 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f89ce6f0cf04373a414d927fa0f8a7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:21.439028 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a52f59-d370-489e-9a2b-2552c3653d32 service cinder] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:21.440136 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a52f59-d370-489e-9a2b-2552c3653d32 service cinder] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:21.440428 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a52f59-d370-489e-9a2b-2552c3653d32 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:21.459413 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a52f59-d370-489e-9a2b-2552c3653d32 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:21.462111 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a52f59-d370-489e-9a2b-2552c3653d32 service cinder] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:21.476460 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 924/1835] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:15:21 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:15:21.480946 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9c7004-d038-4ca5-be75-e545657a09b5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:21.482600 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9c7004-d038-4ca5-be75-e545657a09b5 service cinder] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:21.501233 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 912/1836] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:15:21 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:15:21.603110 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a52f59-d370-489e-9a2b-2552c3653d32 service cinder] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:21.732602 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:15:21.807265 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a52f59-d370-489e-9a2b-2552c3653d32 service cinder] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:21.807507 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9c7004-d038-4ca5-be75-e545657a09b5 service cinder] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:21.813301 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafee590-1d69-4274-af2e-c01ba68b4842 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:21.814272 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafee590-1d69-4274-af2e-c01ba68b4842 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:21.815598 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafee590-1d69-4274-af2e-c01ba68b4842 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:21.815598 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafee590-1d69-4274-af2e-c01ba68b4842 admin admin] PATH_INFO: `/v3/users/5d49c1f43f234bb89df11549d1a57d8e` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:21.817625 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cafee590-1d69-4274-af2e-c01ba68b4842 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5d49c1f43f234bb89df11549d1a57d8e)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:21.820868 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cafee590-1d69-4274-af2e-c01ba68b4842 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:21.822589 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9c7004-d038-4ca5-be75-e545657a09b5 service cinder] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:21.838045 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61941c9f-ceed-4c50-8042-9dd8e3776d9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:21.839253 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61941c9f-ceed-4c50-8042-9dd8e3776d9c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:21.839565 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61941c9f-ceed-4c50-8042-9dd8e3776d9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:21.839852 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61941c9f-ceed-4c50-8042-9dd8e3776d9c admin admin] PATH_INFO: `/v3/users/f3d3cf99393c49abbcb6e0f4b4301997` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:21.843835 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61941c9f-ceed-4c50-8042-9dd8e3776d9c admin admin] RBAC: Authorizing `identity:delete_user(user_id=f3d3cf99393c49abbcb6e0f4b4301997)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:21.845824 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61941c9f-ceed-4c50-8042-9dd8e3776d9c admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:21.863287 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:15:21.899080 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-cafee590-1d69-4274-af2e-c01ba68b4842 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5d49c1f43f234bb89df11549d1a57d8e'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:21.899080 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-cafee590-1d69-4274-af2e-c01ba68b4842 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5d49c1f43f234bb89df11549d1a57d8e'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:21.916151 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-cafee590-1d69-4274-af2e-c01ba68b4842 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5d49c1f43f234bb89df11549d1a57d8e'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:21.928307 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-61941c9f-ceed-4c50-8042-9dd8e3776d9c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f3d3cf99393c49abbcb6e0f4b4301997'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:21.938823 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-61941c9f-ceed-4c50-8042-9dd8e3776d9c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f3d3cf99393c49abbcb6e0f4b4301997'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:21.946966 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-cafee590-1d69-4274-af2e-c01ba68b4842 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5d49c1f43f234bb89df11549d1a57d8e'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:21.960029 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-61941c9f-ceed-4c50-8042-9dd8e3776d9c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f3d3cf99393c49abbcb6e0f4b4301997'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:21.973376 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-cafee590-1d69-4274-af2e-c01ba68b4842 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5d49c1f43f234bb89df11549d1a57d8e'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:21.983127 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 925/1837] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Jul 25 12:15:21 2022] DELETE /identity/v3/users/5d49c1f43f234bb89df11549d1a57d8e => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:15:21.988631 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41554741-b7b7-404c-8f33-595719d061aa None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:21.989042 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41554741-b7b7-404c-8f33-595719d061aa None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:21.989223 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-61941c9f-ceed-4c50-8042-9dd8e3776d9c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f3d3cf99393c49abbcb6e0f4b4301997'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:21.989468 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41554741-b7b7-404c-8f33-595719d061aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:21.990086 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-61941c9f-ceed-4c50-8042-9dd8e3776d9c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f3d3cf99393c49abbcb6e0f4b4301997'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:22.011174 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 913/1838] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Jul 25 12:15:21 2022] DELETE /identity/v3/users/f3d3cf99393c49abbcb6e0f4b4301997 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:15:22.014542 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae94605-08e1-4fc8-9f5e-f3adaaf6d643 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:22.014897 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae94605-08e1-4fc8-9f5e-f3adaaf6d643 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:22.015263 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae94605-08e1-4fc8-9f5e-f3adaaf6d643 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:22.038523 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-41554741-b7b7-404c-8f33-595719d061aa None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:15:22.043322 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-41554741-b7b7-404c-8f33-595719d061aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:15:22.067010 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-aae94605-08e1-4fc8-9f5e-f3adaaf6d643 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:15:22.068364 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-aae94605-08e1-4fc8-9f5e-f3adaaf6d643 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:15:22.188160 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 926/1839] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:15:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 12:15:22.189227 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41554741-b7b7-404c-8f33-595719d061aa None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:22.190761 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-41554741-b7b7-404c-8f33-595719d061aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:15:22.194024 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 914/1840] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:15:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 12:15:22.220544 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae94605-08e1-4fc8-9f5e-f3adaaf6d643 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:22.222842 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-aae94605-08e1-4fc8-9f5e-f3adaaf6d643 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:15:22.297746 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:15:22.303378 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41554741-b7b7-404c-8f33-595719d061aa None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:22.309415 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e8e4de-a058-474a-959f-cc8721aca9ad service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f89ce6f0cf04373a414d927fa0f8a7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:22.310796 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e8e4de-a058-474a-959f-cc8721aca9ad service cinder] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:22.311587 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e8e4de-a058-474a-959f-cc8721aca9ad service cinder] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:22.312017 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e8e4de-a058-474a-959f-cc8721aca9ad service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:22.314149 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-00e8e4de-a058-474a-959f-cc8721aca9ad service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:15:22.315186 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae94605-08e1-4fc8-9f5e-f3adaaf6d643 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:22.321393 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1aa3c68-8127-40ff-a5d4-1fb440b97402 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51733781201243e8abf9eca39b207750', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:22.322211 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1aa3c68-8127-40ff-a5d4-1fb440b97402 service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:22.322503 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1aa3c68-8127-40ff-a5d4-1fb440b97402 service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:22.322887 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1aa3c68-8127-40ff-a5d4-1fb440b97402 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:22.335085 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1aa3c68-8127-40ff-a5d4-1fb440b97402 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:22.336814 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1aa3c68-8127-40ff-a5d4-1fb440b97402 service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:22.350210 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 915/1841] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:15:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:15:22.351833 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1aa3c68-8127-40ff-a5d4-1fb440b97402 service neutron] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:22.362845 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1aa3c68-8127-40ff-a5d4-1fb440b97402 service neutron] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:22.370358 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cba5485-bed8-4371-8a05-640c6a3be629 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51733781201243e8abf9eca39b207750', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:22.371033 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cba5485-bed8-4371-8a05-640c6a3be629 service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:22.371321 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cba5485-bed8-4371-8a05-640c6a3be629 service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:22.371739 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cba5485-bed8-4371-8a05-640c6a3be629 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:22.384326 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cba5485-bed8-4371-8a05-640c6a3be629 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:22.385763 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cba5485-bed8-4371-8a05-640c6a3be629 service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:22.395914 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e8e4de-a058-474a-959f-cc8721aca9ad service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:22.399050 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 916/1842] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:15:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:15:22.399197 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e8e4de-a058-474a-959f-cc8721aca9ad service cinder] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:22.408483 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 927/1843] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:15:22 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:15:23.298851 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cba5485-bed8-4371-8a05-640c6a3be629 service neutron] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:23.301335 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-5cba5485-bed8-4371-8a05-640c6a3be629 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:15:23.343534 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cba5485-bed8-4371-8a05-640c6a3be629 service neutron] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:23.353581 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36971cc6-aea3-40d3-a7ec-1d8ffd9db71a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:23.354671 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36971cc6-aea3-40d3-a7ec-1d8ffd9db71a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:23.354866 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36971cc6-aea3-40d3-a7ec-1d8ffd9db71a admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:23.355140 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36971cc6-aea3-40d3-a7ec-1d8ffd9db71a admin admin] PATH_INFO: `/v3/projects/fc3f1b387a2f4119aa5adbed8181c26a` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:23.356398 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36971cc6-aea3-40d3-a7ec-1d8ffd9db71a admin admin] RBAC: Authorizing `identity:delete_project(project_id=fc3f1b387a2f4119aa5adbed8181c26a)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:23.358317 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36971cc6-aea3-40d3-a7ec-1d8ffd9db71a admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:23.373172 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:15:23.426580 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-36971cc6-aea3-40d3-a7ec-1d8ffd9db71a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fc3f1b387a2f4119aa5adbed8181c26a', 'request_id': 'req-36971cc6-aea3-40d3-a7ec-1d8ffd9db71a'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:23.441130 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-36971cc6-aea3-40d3-a7ec-1d8ffd9db71a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fc3f1b387a2f4119aa5adbed8181c26a', 'request_id': 'req-36971cc6-aea3-40d3-a7ec-1d8ffd9db71a'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:23.452166 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-36971cc6-aea3-40d3-a7ec-1d8ffd9db71a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fc3f1b387a2f4119aa5adbed8181c26a', 'request_id': 'req-36971cc6-aea3-40d3-a7ec-1d8ffd9db71a'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:23.469860 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-36971cc6-aea3-40d3-a7ec-1d8ffd9db71a admin admin] The token cache is being invalidate because project fc3f1b387a2f4119aa5adbed8181c26a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 12:15:23.469860 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-36971cc6-aea3-40d3-a7ec-1d8ffd9db71a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:23.469860 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-36971cc6-aea3-40d3-a7ec-1d8ffd9db71a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:23.471641 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 917/1844] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Jul 25 12:15:23 2022] DELETE /identity/v3/projects/fc3f1b387a2f4119aa5adbed8181c26a => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:15:23.476048 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e8e4de-a058-474a-959f-cc8721aca9ad service cinder] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:23.492667 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e8e4de-a058-474a-959f-cc8721aca9ad service cinder] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:23.501862 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36971cc6-aea3-40d3-a7ec-1d8ffd9db71a admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:23.503384 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-36971cc6-aea3-40d3-a7ec-1d8ffd9db71a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:15:23.637996 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28fe5f4-cd94-49ac-a7f0-8e9f88be01ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:23.639147 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28fe5f4-cd94-49ac-a7f0-8e9f88be01ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:23.640470 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28fe5f4-cd94-49ac-a7f0-8e9f88be01ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:23.641643 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28fe5f4-cd94-49ac-a7f0-8e9f88be01ee admin admin] PATH_INFO: `/v3/users/d2e670c4ab554a0e82b219e6a99ad025` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:23.643884 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28fe5f4-cd94-49ac-a7f0-8e9f88be01ee admin admin] RBAC: Authorizing `identity:delete_user(user_id=d2e670c4ab554a0e82b219e6a99ad025)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:23.646819 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28fe5f4-cd94-49ac-a7f0-8e9f88be01ee admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:23.661559 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36971cc6-aea3-40d3-a7ec-1d8ffd9db71a admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:23.670749 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ba8ef8-6ae3-41a0-8aa8-d44d51d24f71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:23.671667 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ba8ef8-6ae3-41a0-8aa8-d44d51d24f71 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:23.671975 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ba8ef8-6ae3-41a0-8aa8-d44d51d24f71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:23.672245 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ba8ef8-6ae3-41a0-8aa8-d44d51d24f71 admin admin] PATH_INFO: `/v3/projects/0ce09e226a444016b47e4e286061e66f` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:23.673464 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ba8ef8-6ae3-41a0-8aa8-d44d51d24f71 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0ce09e226a444016b47e4e286061e66f)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:23.675385 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ba8ef8-6ae3-41a0-8aa8-d44d51d24f71 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:23.712898 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-f28fe5f4-cd94-49ac-a7f0-8e9f88be01ee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd2e670c4ab554a0e82b219e6a99ad025'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:23.713742 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-f28fe5f4-cd94-49ac-a7f0-8e9f88be01ee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd2e670c4ab554a0e82b219e6a99ad025'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:23.722390 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-f28fe5f4-cd94-49ac-a7f0-8e9f88be01ee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd2e670c4ab554a0e82b219e6a99ad025'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:23.740857 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-f28fe5f4-cd94-49ac-a7f0-8e9f88be01ee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd2e670c4ab554a0e82b219e6a99ad025'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:23.744070 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-d5ba8ef8-6ae3-41a0-8aa8-d44d51d24f71 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0ce09e226a444016b47e4e286061e66f', 'request_id': 'req-d5ba8ef8-6ae3-41a0-8aa8-d44d51d24f71'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:23.750931 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-f28fe5f4-cd94-49ac-a7f0-8e9f88be01ee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd2e670c4ab554a0e82b219e6a99ad025'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:23.759765 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 928/1845] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Jul 25 12:15:23 2022] DELETE /identity/v3/users/d2e670c4ab554a0e82b219e6a99ad025 => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 12:15:23.765103 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-d5ba8ef8-6ae3-41a0-8aa8-d44d51d24f71 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0ce09e226a444016b47e4e286061e66f', 'request_id': 'req-d5ba8ef8-6ae3-41a0-8aa8-d44d51d24f71'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:23.776567 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-d5ba8ef8-6ae3-41a0-8aa8-d44d51d24f71 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0ce09e226a444016b47e4e286061e66f', 'request_id': 'req-d5ba8ef8-6ae3-41a0-8aa8-d44d51d24f71'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:23.794363 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-d5ba8ef8-6ae3-41a0-8aa8-d44d51d24f71 admin admin] The token cache is being invalidate because project 0ce09e226a444016b47e4e286061e66f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 12:15:23.795071 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-d5ba8ef8-6ae3-41a0-8aa8-d44d51d24f71 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:23.795714 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-d5ba8ef8-6ae3-41a0-8aa8-d44d51d24f71 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:23.802415 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28fe5f4-cd94-49ac-a7f0-8e9f88be01ee admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:23.803084 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 918/1846] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Jul 25 12:15:23 2022] DELETE /identity/v3/projects/0ce09e226a444016b47e4e286061e66f => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 12:15:23.805432 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-f28fe5f4-cd94-49ac-a7f0-8e9f88be01ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:15:23.853747 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:15:23.995802 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28fe5f4-cd94-49ac-a7f0-8e9f88be01ee admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:24.004676 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15eeac84-e0e1-41b9-b7b8-27eb1c590fdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:24.005923 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15eeac84-e0e1-41b9-b7b8-27eb1c590fdb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:24.008703 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15eeac84-e0e1-41b9-b7b8-27eb1c590fdb admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:24.009097 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15eeac84-e0e1-41b9-b7b8-27eb1c590fdb admin admin] PATH_INFO: `/v3/users/8a9da76d97524aeca137c2ae26396f3f` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:24.041043 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15eeac84-e0e1-41b9-b7b8-27eb1c590fdb admin admin] RBAC: Authorizing `identity:delete_user(user_id=8a9da76d97524aeca137c2ae26396f3f)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:24.044105 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15eeac84-e0e1-41b9-b7b8-27eb1c590fdb admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:24.154546 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-15eeac84-e0e1-41b9-b7b8-27eb1c590fdb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8a9da76d97524aeca137c2ae26396f3f'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:24.155838 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-15eeac84-e0e1-41b9-b7b8-27eb1c590fdb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8a9da76d97524aeca137c2ae26396f3f'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:24.170003 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-15eeac84-e0e1-41b9-b7b8-27eb1c590fdb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8a9da76d97524aeca137c2ae26396f3f'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:24.198028 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-15eeac84-e0e1-41b9-b7b8-27eb1c590fdb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8a9da76d97524aeca137c2ae26396f3f'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:24.234770 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-15eeac84-e0e1-41b9-b7b8-27eb1c590fdb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8a9da76d97524aeca137c2ae26396f3f'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:24.276753 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 929/1847] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Jul 25 12:15:23 2022] DELETE /identity/v3/users/8a9da76d97524aeca137c2ae26396f3f => generated 0 bytes in 472 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 12:15:24.855050 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ba8ef8-6ae3-41a0-8aa8-d44d51d24f71 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:24.856220 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-d5ba8ef8-6ae3-41a0-8aa8-d44d51d24f71 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:15:24.856544 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15eeac84-e0e1-41b9-b7b8-27eb1c590fdb admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:24.858052 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-15eeac84-e0e1-41b9-b7b8-27eb1c590fdb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:15:24.926967 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:15:24.945092 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15eeac84-e0e1-41b9-b7b8-27eb1c590fdb admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:24.949858 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e241044-b578-4ce4-bdb9-50aace9b2b0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:24.950731 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e241044-b578-4ce4-bdb9-50aace9b2b0c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:24.951014 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e241044-b578-4ce4-bdb9-50aace9b2b0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:24.951294 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e241044-b578-4ce4-bdb9-50aace9b2b0c admin admin] PATH_INFO: `/v3/projects/3798ab22a9004d1ca57c0ddfe91991e7` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:24.952527 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e241044-b578-4ce4-bdb9-50aace9b2b0c admin admin] RBAC: Authorizing `identity:delete_project(project_id=3798ab22a9004d1ca57c0ddfe91991e7)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:24.954458 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e241044-b578-4ce4-bdb9-50aace9b2b0c admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:24.964936 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ba8ef8-6ae3-41a0-8aa8-d44d51d24f71 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:24.969683 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c94b762-cdf4-4e91-988f-583445775d06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:24.970552 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c94b762-cdf4-4e91-988f-583445775d06 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:24.970841 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c94b762-cdf4-4e91-988f-583445775d06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:24.971126 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c94b762-cdf4-4e91-988f-583445775d06 admin admin] PATH_INFO: `/v3/projects/8fa9b873c09d4eea9feaaf3973b6238f` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:24.972350 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c94b762-cdf4-4e91-988f-583445775d06 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8fa9b873c09d4eea9feaaf3973b6238f)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:24.974256 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c94b762-cdf4-4e91-988f-583445775d06 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:25.000994 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:15:25.005144 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-8e241044-b578-4ce4-bdb9-50aace9b2b0c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3798ab22a9004d1ca57c0ddfe91991e7', 'request_id': 'req-8e241044-b578-4ce4-bdb9-50aace9b2b0c'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:25.012686 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-8e241044-b578-4ce4-bdb9-50aace9b2b0c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3798ab22a9004d1ca57c0ddfe91991e7', 'request_id': 'req-8e241044-b578-4ce4-bdb9-50aace9b2b0c'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:25.018891 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-8e241044-b578-4ce4-bdb9-50aace9b2b0c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3798ab22a9004d1ca57c0ddfe91991e7', 'request_id': 'req-8e241044-b578-4ce4-bdb9-50aace9b2b0c'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:25.024618 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-3c94b762-cdf4-4e91-988f-583445775d06 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8fa9b873c09d4eea9feaaf3973b6238f', 'request_id': 'req-3c94b762-cdf4-4e91-988f-583445775d06'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:25.031208 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-8e241044-b578-4ce4-bdb9-50aace9b2b0c admin admin] The token cache is being invalidate because project 3798ab22a9004d1ca57c0ddfe91991e7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80298) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 12:15:25.031595 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-8e241044-b578-4ce4-bdb9-50aace9b2b0c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:25.031815 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-3c94b762-cdf4-4e91-988f-583445775d06 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8fa9b873c09d4eea9feaaf3973b6238f', 'request_id': 'req-3c94b762-cdf4-4e91-988f-583445775d06'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:25.032372 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-8e241044-b578-4ce4-bdb9-50aace9b2b0c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:25.034213 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 930/1848] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Jul 25 12:15:24 2022] DELETE /identity/v3/projects/3798ab22a9004d1ca57c0ddfe91991e7 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:15:25.039438 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50d254d-a02a-4ccc-8ce4-c9d5e63363b9 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:25.039695 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-3c94b762-cdf4-4e91-988f-583445775d06 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8fa9b873c09d4eea9feaaf3973b6238f', 'request_id': 'req-3c94b762-cdf4-4e91-988f-583445775d06'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:25.039804 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50d254d-a02a-4ccc-8ce4-c9d5e63363b9 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:25.040046 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50d254d-a02a-4ccc-8ce4-c9d5e63363b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:25.053459 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-3c94b762-cdf4-4e91-988f-583445775d06 admin admin] The token cache is being invalidate because project 8fa9b873c09d4eea9feaaf3973b6238f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 12:15:25.053782 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-3c94b762-cdf4-4e91-988f-583445775d06 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:25.055466 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-3c94b762-cdf4-4e91-988f-583445775d06 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:25.057143 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 919/1849] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Jul 25 12:15:24 2022] DELETE /identity/v3/projects/8fa9b873c09d4eea9feaaf3973b6238f => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:15:25.061799 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-c50d254d-a02a-4ccc-8ce4-c9d5e63363b9 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:15:25.061938 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d9dddb-82de-41a2-978b-0f3228d10343 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:25.062125 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d9dddb-82de-41a2-978b-0f3228d10343 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:25.062410 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d9dddb-82de-41a2-978b-0f3228d10343 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:25.063007 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-c50d254d-a02a-4ccc-8ce4-c9d5e63363b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:15:25.083828 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-d1d9dddb-82de-41a2-978b-0f3228d10343 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:15:25.085078 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-d1d9dddb-82de-41a2-978b-0f3228d10343 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:15:25.133500 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 931/1850] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:15:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 12:15:25.141991 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b757af68-d914-4112-b946-26929d8c9a5c None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:25.142419 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b757af68-d914-4112-b946-26929d8c9a5c None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:25.142711 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b757af68-d914-4112-b946-26929d8c9a5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:25.168862 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 920/1851] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:15:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 12:15:25.169266 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-b757af68-d914-4112-b946-26929d8c9a5c None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:15:25.169940 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-b757af68-d914-4112-b946-26929d8c9a5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:15:25.170433 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9412d2fa-c765-4ff1-ad39-7535a2fc890e None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:25.170872 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9412d2fa-c765-4ff1-ad39-7535a2fc890e None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:25.171172 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9412d2fa-c765-4ff1-ad39-7535a2fc890e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:25.184482 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 932/1852] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:15:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 12:15:25.191425 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b757af68-d914-4112-b946-26929d8c9a5c None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:25.213006 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b757af68-d914-4112-b946-26929d8c9a5c None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:25.220797 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517eff39-0e87-4bf5-aa9c-48ac44387a1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:25.222036 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517eff39-0e87-4bf5-aa9c-48ac44387a1f admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:25.222561 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-9412d2fa-c765-4ff1-ad39-7535a2fc890e None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:15:25.223605 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517eff39-0e87-4bf5-aa9c-48ac44387a1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:25.223903 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-9412d2fa-c765-4ff1-ad39-7535a2fc890e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:15:25.224293 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517eff39-0e87-4bf5-aa9c-48ac44387a1f admin admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:25.227505 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517eff39-0e87-4bf5-aa9c-48ac44387a1f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:15:25.228106 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517eff39-0e87-4bf5-aa9c-48ac44387a1f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:25.234760 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517eff39-0e87-4bf5-aa9c-48ac44387a1f admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:25.236141 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 921/1853] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:15:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 12:15:25.240244 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9412d2fa-c765-4ff1-ad39-7535a2fc890e None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:25.254519 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9412d2fa-c765-4ff1-ad39-7535a2fc890e None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:25.261717 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e81bbe5-2636-4ef3-83bc-8596ffde19d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:25.262421 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 933/1854] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Jul 25 12:15:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:15:25.266879 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e81bbe5-2636-4ef3-83bc-8596ffde19d9 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:25.267096 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517eff39-0e87-4bf5-aa9c-48ac44387a1f admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:25.267230 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e81bbe5-2636-4ef3-83bc-8596ffde19d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:25.267648 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e81bbe5-2636-4ef3-83bc-8596ffde19d9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:25.268456 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e81bbe5-2636-4ef3-83bc-8596ffde19d9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:15:25.268770 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e81bbe5-2636-4ef3-83bc-8596ffde19d9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:25.270606 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e81bbe5-2636-4ef3-83bc-8596ffde19d9 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:25.292950 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 922/1855] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Jul 25 12:15:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 12:15:25.293559 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517eff39-0e87-4bf5-aa9c-48ac44387a1f admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:25.296650 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e81bbe5-2636-4ef3-83bc-8596ffde19d9 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:25.304713 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8f5710-178e-4706-a1f5-51928cd72057 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:25.305405 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8f5710-178e-4706-a1f5-51928cd72057 admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:25.306016 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8f5710-178e-4706-a1f5-51928cd72057 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:25.306016 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8f5710-178e-4706-a1f5-51928cd72057 admin admin] PATH_INFO: `/v3/projects` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:25.306872 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8f5710-178e-4706-a1f5-51928cd72057 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:25.311084 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8f5710-178e-4706-a1f5-51928cd72057 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:25.320393 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e81bbe5-2636-4ef3-83bc-8596ffde19d9 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:25.330699 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947df2f1-98fc-4ee4-87ea-99a17da14477 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:25.335694 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947df2f1-98fc-4ee4-87ea-99a17da14477 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:25.336128 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947df2f1-98fc-4ee4-87ea-99a17da14477 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:25.336447 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947df2f1-98fc-4ee4-87ea-99a17da14477 admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:25.337258 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947df2f1-98fc-4ee4-87ea-99a17da14477 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:25.339994 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947df2f1-98fc-4ee4-87ea-99a17da14477 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:25.362216 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 934/1856] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Jul 25 12:15:25 2022] POST /identity/v3/projects => generated 379 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 12:15:25.363420 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8f5710-178e-4706-a1f5-51928cd72057 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:25.419659 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 923/1857] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Jul 25 12:15:25 2022] POST /identity/v3/projects => generated 387 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:15:25.422350 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947df2f1-98fc-4ee4-87ea-99a17da14477 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:25.469682 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:15:25.504897 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8f5710-178e-4706-a1f5-51928cd72057 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:25.580611 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947df2f1-98fc-4ee4-87ea-99a17da14477 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:25.597279 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627471c6-cbab-4637-b4dc-6fb32b807253 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:25.606385 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627471c6-cbab-4637-b4dc-6fb32b807253 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:25.606752 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627471c6-cbab-4637-b4dc-6fb32b807253 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:25.606969 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627471c6-cbab-4637-b4dc-6fb32b807253 admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:25.609068 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=94,keystone:DELETE=4,keystone:INSERT=2 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:15:25.609299 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627471c6-cbab-4637-b4dc-6fb32b807253 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:25.615688 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627471c6-cbab-4637-b4dc-6fb32b807253 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:25.698580 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 924/1858] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Jul 25 12:15:25 2022] POST /identity/v3/users => generated 294 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 12:15:25.701143 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a50621a-af8f-4d5c-9802-6f27945b2d47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:25.701850 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a50621a-af8f-4d5c-9802-6f27945b2d47 admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:25.702060 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627471c6-cbab-4637-b4dc-6fb32b807253 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:25.704339 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a50621a-af8f-4d5c-9802-6f27945b2d47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:25.706292 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a50621a-af8f-4d5c-9802-6f27945b2d47 admin admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:25.719297 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a50621a-af8f-4d5c-9802-6f27945b2d47 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:25.721528 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a50621a-af8f-4d5c-9802-6f27945b2d47 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:25.741854 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627471c6-cbab-4637-b4dc-6fb32b807253 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:25.757895 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35f4d63-9a48-46f5-a4f5-39b8a8db3855 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:25.758868 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35f4d63-9a48-46f5-a4f5-39b8a8db3855 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:25.759406 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35f4d63-9a48-46f5-a4f5-39b8a8db3855 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:25.760039 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35f4d63-9a48-46f5-a4f5-39b8a8db3855 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:25.762434 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35f4d63-9a48-46f5-a4f5-39b8a8db3855 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:25.767214 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35f4d63-9a48-46f5-a4f5-39b8a8db3855 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:25.774977 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 935/1859] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Jul 25 12:15:25 2022] POST /identity/v3/users => generated 290 bytes in 412 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:15:25.783960 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a50621a-af8f-4d5c-9802-6f27945b2d47 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:25.812390 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 925/1860] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Jul 25 12:15:25 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:15:25.816918 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35f4d63-9a48-46f5-a4f5-39b8a8db3855 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:25.818048 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a50621a-af8f-4d5c-9802-6f27945b2d47 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:25.833077 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f623f09-16d0-49db-9707-826bd6c7139d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:25.835338 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f623f09-16d0-49db-9707-826bd6c7139d admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:25.835657 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f623f09-16d0-49db-9707-826bd6c7139d admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:25.837066 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f623f09-16d0-49db-9707-826bd6c7139d admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:25.840919 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35f4d63-9a48-46f5-a4f5-39b8a8db3855 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:25.841184 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f623f09-16d0-49db-9707-826bd6c7139d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:25.847078 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a7de8a-3f6f-4a28-84e2-f8b3910f20ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:25.848346 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f623f09-16d0-49db-9707-826bd6c7139d admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:25.854823 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a7de8a-3f6f-4a28-84e2-f8b3910f20ed admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:25.855579 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a7de8a-3f6f-4a28-84e2-f8b3910f20ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:25.856769 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a7de8a-3f6f-4a28-84e2-f8b3910f20ed admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:25.857741 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a7de8a-3f6f-4a28-84e2-f8b3910f20ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:25.859534 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a7de8a-3f6f-4a28-84e2-f8b3910f20ed admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:25.875243 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 936/1861] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Jul 25 12:15:25 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:15:25.876763 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f623f09-16d0-49db-9707-826bd6c7139d admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:25.892794 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 926/1862] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Jul 25 12:15:25 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:15:25.894556 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a7de8a-3f6f-4a28-84e2-f8b3910f20ed admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:25.909645 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f623f09-16d0-49db-9707-826bd6c7139d admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:25.929862 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a7de8a-3f6f-4a28-84e2-f8b3910f20ed admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:25.932085 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab54fe3-cfb3-48fb-84f3-3dd62554d4a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:25.933171 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab54fe3-cfb3-48fb-84f3-3dd62554d4a6 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:25.933565 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab54fe3-cfb3-48fb-84f3-3dd62554d4a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:25.934454 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab54fe3-cfb3-48fb-84f3-3dd62554d4a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:25.936876 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab54fe3-cfb3-48fb-84f3-3dd62554d4a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:25.943456 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab54fe3-cfb3-48fb-84f3-3dd62554d4a6 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:25.946368 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ff0059-3c0f-4fd0-a977-0b4758783295 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:25.949560 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ff0059-3c0f-4fd0-a977-0b4758783295 admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:25.950499 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ff0059-3c0f-4fd0-a977-0b4758783295 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:25.950891 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ff0059-3c0f-4fd0-a977-0b4758783295 admin admin] PATH_INFO: `/v3/projects/c9c48b754f9341e0b2c415623ccdf9b6/users/b2fe4b0d2f024c16b3a2aa0884453ea9/roles/472313ae69174578824df0bed6390744` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:25.982767 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 937/1863] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Jul 25 12:15:25 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:15:25.984398 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab54fe3-cfb3-48fb-84f3-3dd62554d4a6 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:26.005157 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ff0059-3c0f-4fd0-a977-0b4758783295 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c9c48b754f9341e0b2c415623ccdf9b6, user_id=b2fe4b0d2f024c16b3a2aa0884453ea9, role_id=472313ae69174578824df0bed6390744)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:26.013105 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ff0059-3c0f-4fd0-a977-0b4758783295 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:26.017192 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab54fe3-cfb3-48fb-84f3-3dd62554d4a6 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:26.033416 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e854127-e8d1-4aba-9c2b-3620ccb3147e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:26.037235 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e854127-e8d1-4aba-9c2b-3620ccb3147e admin admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:26.037828 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e854127-e8d1-4aba-9c2b-3620ccb3147e admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:26.038881 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e854127-e8d1-4aba-9c2b-3620ccb3147e admin admin] PATH_INFO: `/v3/projects/c86a86619d764987bbb889ca894dbc50/users/f7049d550b83487ea633f231e986cd66/roles/472313ae69174578824df0bed6390744` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:26.056061 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 927/1864] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Jul 25 12:15:25 2022] PUT /identity/v3/projects/c9c48b754f9341e0b2c415623ccdf9b6/users/b2fe4b0d2f024c16b3a2aa0884453ea9/roles/472313ae69174578824df0bed6390744 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:15:26.067077 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd1d66a-9e5b-475c-b0bb-6fb67d031855 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:26.067532 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd1d66a-9e5b-475c-b0bb-6fb67d031855 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:26.067937 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd1d66a-9e5b-475c-b0bb-6fb67d031855 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:26.079323 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e854127-e8d1-4aba-9c2b-3620ccb3147e admin admin] RBAC: Authorizing `identity:create_grant(project_id=c86a86619d764987bbb889ca894dbc50, user_id=f7049d550b83487ea633f231e986cd66, role_id=472313ae69174578824df0bed6390744)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:26.079323 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e854127-e8d1-4aba-9c2b-3620ccb3147e admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:26.097945 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 938/1865] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Jul 25 12:15:25 2022] PUT /identity/v3/projects/c86a86619d764987bbb889ca894dbc50/users/f7049d550b83487ea633f231e986cd66/roles/472313ae69174578824df0bed6390744 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:15:26.110103 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c682d0c-c47a-48aa-a4ca-ff591de21f56 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:26.110391 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c682d0c-c47a-48aa-a4ca-ff591de21f56 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:26.110752 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c682d0c-c47a-48aa-a4ca-ff591de21f56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:26.144214 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-4cd1d66a-9e5b-475c-b0bb-6fb67d031855 None None] MFA Rules not processed for user `b2fe4b0d2f024c16b3a2aa0884453ea9`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:15:26.144214 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-4cd1d66a-9e5b-475c-b0bb-6fb67d031855 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:15:26.170517 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-5c682d0c-c47a-48aa-a4ca-ff591de21f56 None None] MFA Rules not processed for user `f7049d550b83487ea633f231e986cd66`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:15:26.171949 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-5c682d0c-c47a-48aa-a4ca-ff591de21f56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:15:26.326692 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 928/1866] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:15:26 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 12:15:26.331260 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd1d66a-9e5b-475c-b0bb-6fb67d031855 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:26.338467 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 939/1867] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:15:26 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:15:26.348030 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c682d0c-c47a-48aa-a4ca-ff591de21f56 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:26.435556 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:15:26.462439 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd1d66a-9e5b-475c-b0bb-6fb67d031855 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:26.473338 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c682d0c-c47a-48aa-a4ca-ff591de21f56 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:26.474504 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1663022e-e58d-4670-ac49-0ea4299efe30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:26.475302 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1663022e-e58d-4670-ac49-0ea4299efe30 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:26.476174 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1663022e-e58d-4670-ac49-0ea4299efe30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:26.476451 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1663022e-e58d-4670-ac49-0ea4299efe30 admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:26.477378 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1663022e-e58d-4670-ac49-0ea4299efe30 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:26.479753 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1663022e-e58d-4670-ac49-0ea4299efe30 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:26.486699 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ac7905-c48b-4ecb-a87a-5769be1a4aeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:26.487387 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ac7905-c48b-4ecb-a87a-5769be1a4aeb admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:26.487704 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ac7905-c48b-4ecb-a87a-5769be1a4aeb admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:26.487989 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ac7905-c48b-4ecb-a87a-5769be1a4aeb admin admin] PATH_INFO: `/v3/projects` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:26.488695 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ac7905-c48b-4ecb-a87a-5769be1a4aeb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:26.492457 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ac7905-c48b-4ecb-a87a-5769be1a4aeb admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:26.521230 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:15:26.554928 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 929/1868] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Jul 25 12:15:26 2022] POST /identity/v3/projects => generated 387 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 12:15:26.558641 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1663022e-e58d-4670-ac49-0ea4299efe30 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:26.562567 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 940/1869] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Jul 25 12:15:26 2022] POST /identity/v3/projects => generated 381 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:15:26.563794 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ac7905-c48b-4ecb-a87a-5769be1a4aeb admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:26.685844 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ac7905-c48b-4ecb-a87a-5769be1a4aeb admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:26.692617 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1663022e-e58d-4670-ac49-0ea4299efe30 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:26.693843 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8f313d-e891-4ac1-b4cd-fc4d28c4b3f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:26.695166 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8f313d-e891-4ac1-b4cd-fc4d28c4b3f1 admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:26.695457 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8f313d-e891-4ac1-b4cd-fc4d28c4b3f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:26.695764 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8f313d-e891-4ac1-b4cd-fc4d28c4b3f1 admin admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:26.696618 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8f313d-e891-4ac1-b4cd-fc4d28c4b3f1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:26.698596 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385a006f-95a7-4c81-b9d2-4b0eb294943d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:26.699404 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8f313d-e891-4ac1-b4cd-fc4d28c4b3f1 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:26.699532 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385a006f-95a7-4c81-b9d2-4b0eb294943d admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:26.699837 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385a006f-95a7-4c81-b9d2-4b0eb294943d admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:26.700146 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385a006f-95a7-4c81-b9d2-4b0eb294943d admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:26.700879 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385a006f-95a7-4c81-b9d2-4b0eb294943d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:26.702937 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385a006f-95a7-4c81-b9d2-4b0eb294943d admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:26.738938 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 941/1870] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Jul 25 12:15:26 2022] POST /identity/v3/users => generated 297 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:15:26.744248 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 930/1871] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Jul 25 12:15:26 2022] POST /identity/v3/users => generated 300 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:15:26.744400 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8f313d-e891-4ac1-b4cd-fc4d28c4b3f1 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:26.745341 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385a006f-95a7-4c81-b9d2-4b0eb294943d admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:26.754488 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8f313d-e891-4ac1-b4cd-fc4d28c4b3f1 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:26.761425 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29dbc10e-c5b3-4c28-96df-b5fcfdcb08e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:26.761700 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dbc10e-c5b3-4c28-96df-b5fcfdcb08e4 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:26.762048 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dbc10e-c5b3-4c28-96df-b5fcfdcb08e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:26.762362 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dbc10e-c5b3-4c28-96df-b5fcfdcb08e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:26.763349 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29dbc10e-c5b3-4c28-96df-b5fcfdcb08e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:26.765016 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29dbc10e-c5b3-4c28-96df-b5fcfdcb08e4 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:26.769465 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385a006f-95a7-4c81-b9d2-4b0eb294943d admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:26.778572 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 942/1872] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Jul 25 12:15:26 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:15:26.779068 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11178610-015f-4003-98af-723164303670 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:26.780034 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11178610-015f-4003-98af-723164303670 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:26.780572 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11178610-015f-4003-98af-723164303670 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:26.781147 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11178610-015f-4003-98af-723164303670 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:26.782135 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11178610-015f-4003-98af-723164303670 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:26.783021 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29dbc10e-c5b3-4c28-96df-b5fcfdcb08e4 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:26.784680 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11178610-015f-4003-98af-723164303670 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:26.797558 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 931/1873] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Jul 25 12:15:26 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 12:15:26.800784 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29dbc10e-c5b3-4c28-96df-b5fcfdcb08e4 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:26.801050 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11178610-015f-4003-98af-723164303670 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:26.814904 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11178610-015f-4003-98af-723164303670 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:26.817219 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc5df1e-4d67-481a-93e0-70642ef4d247 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:26.818143 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc5df1e-4d67-481a-93e0-70642ef4d247 admin admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:26.818432 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc5df1e-4d67-481a-93e0-70642ef4d247 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:26.818752 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc5df1e-4d67-481a-93e0-70642ef4d247 admin admin] PATH_INFO: `/v3/projects/fcb6e4f7ec084ec98c46d8641aecb597/users/85ddb776c9fc4d8292c92eaf88bc43a5/roles/a7d20885b6d94928a34b3cc5b3a00006` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:26.825111 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b24e044-b064-4d94-9a14-2088d4a13eb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:26.826011 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b24e044-b064-4d94-9a14-2088d4a13eb9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:26.826297 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b24e044-b064-4d94-9a14-2088d4a13eb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:26.826578 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b24e044-b064-4d94-9a14-2088d4a13eb9 admin admin] PATH_INFO: `/v3/projects/3d38f98edb0a486199cae6655eeb3ed6/users/2ca0b9452f774b01a727942017e25628/roles/a7d20885b6d94928a34b3cc5b3a00006` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:26.839038 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc5df1e-4d67-481a-93e0-70642ef4d247 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fcb6e4f7ec084ec98c46d8641aecb597, user_id=85ddb776c9fc4d8292c92eaf88bc43a5, role_id=a7d20885b6d94928a34b3cc5b3a00006)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:26.841883 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc5df1e-4d67-481a-93e0-70642ef4d247 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:26.843878 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b24e044-b064-4d94-9a14-2088d4a13eb9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3d38f98edb0a486199cae6655eeb3ed6, user_id=2ca0b9452f774b01a727942017e25628, role_id=a7d20885b6d94928a34b3cc5b3a00006)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:26.846653 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b24e044-b064-4d94-9a14-2088d4a13eb9 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:26.857547 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 943/1874] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Jul 25 12:15:26 2022] PUT /identity/v3/projects/fcb6e4f7ec084ec98c46d8641aecb597/users/85ddb776c9fc4d8292c92eaf88bc43a5/roles/a7d20885b6d94928a34b3cc5b3a00006 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:15:26.859642 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 932/1875] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Jul 25 12:15:26 2022] PUT /identity/v3/projects/3d38f98edb0a486199cae6655eeb3ed6/users/2ca0b9452f774b01a727942017e25628/roles/a7d20885b6d94928a34b3cc5b3a00006 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:15:26.861216 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc5df1e-4d67-481a-93e0-70642ef4d247 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:26.863141 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b24e044-b064-4d94-9a14-2088d4a13eb9 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:26.933684 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc5df1e-4d67-481a-93e0-70642ef4d247 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:26.938234 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76c21ba-4aa6-48cd-abca-859013d81b35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:26.938905 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76c21ba-4aa6-48cd-abca-859013d81b35 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:26.939195 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76c21ba-4aa6-48cd-abca-859013d81b35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:26.939481 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76c21ba-4aa6-48cd-abca-859013d81b35 admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:26.940254 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76c21ba-4aa6-48cd-abca-859013d81b35 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:26.941647 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b24e044-b064-4d94-9a14-2088d4a13eb9 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:26.941852 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76c21ba-4aa6-48cd-abca-859013d81b35 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:26.947090 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab1bc84-43e1-4f56-ac9c-d398cff3f3e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:26.947799 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab1bc84-43e1-4f56-ac9c-d398cff3f3e9 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:26.948090 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab1bc84-43e1-4f56-ac9c-d398cff3f3e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:26.948375 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab1bc84-43e1-4f56-ac9c-d398cff3f3e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:26.949113 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab1bc84-43e1-4f56-ac9c-d398cff3f3e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:26.950747 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab1bc84-43e1-4f56-ac9c-d398cff3f3e9 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:26.955369 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 944/1876] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Jul 25 12:15:26 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 12:15:26.956849 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76c21ba-4aa6-48cd-abca-859013d81b35 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:26.964206 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 933/1877] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Jul 25 12:15:26 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 12:15:26.965556 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab1bc84-43e1-4f56-ac9c-d398cff3f3e9 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:26.967407 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76c21ba-4aa6-48cd-abca-859013d81b35 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:26.971818 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0efea5-348e-401b-bb2c-146e23d37f21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:26.973069 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0efea5-348e-401b-bb2c-146e23d37f21 admin admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:26.973069 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0efea5-348e-401b-bb2c-146e23d37f21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:26.973255 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0efea5-348e-401b-bb2c-146e23d37f21 admin admin] PATH_INFO: `/v3/projects/fcb6e4f7ec084ec98c46d8641aecb597/users/85ddb776c9fc4d8292c92eaf88bc43a5/roles/a7d20885b6d94928a34b3cc5b3a00006` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:26.976948 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0efea5-348e-401b-bb2c-146e23d37f21 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fcb6e4f7ec084ec98c46d8641aecb597, user_id=85ddb776c9fc4d8292c92eaf88bc43a5, role_id=a7d20885b6d94928a34b3cc5b3a00006)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:26.977350 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab1bc84-43e1-4f56-ac9c-d398cff3f3e9 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:26.979679 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0efea5-348e-401b-bb2c-146e23d37f21 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:26.982197 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850422d4-acaa-4e1e-bf29-51998d8687ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:26.983212 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850422d4-acaa-4e1e-bf29-51998d8687ac admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:26.983637 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850422d4-acaa-4e1e-bf29-51998d8687ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:26.984024 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850422d4-acaa-4e1e-bf29-51998d8687ac admin admin] PATH_INFO: `/v3/projects/3d38f98edb0a486199cae6655eeb3ed6/users/2ca0b9452f774b01a727942017e25628/roles/a7d20885b6d94928a34b3cc5b3a00006` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:26.986253 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850422d4-acaa-4e1e-bf29-51998d8687ac admin admin] RBAC: Authorizing `identity:create_grant(project_id=3d38f98edb0a486199cae6655eeb3ed6, user_id=2ca0b9452f774b01a727942017e25628, role_id=a7d20885b6d94928a34b3cc5b3a00006)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:26.989145 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850422d4-acaa-4e1e-bf29-51998d8687ac admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:26.992883 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 945/1878] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Jul 25 12:15:26 2022] PUT /identity/v3/projects/fcb6e4f7ec084ec98c46d8641aecb597/users/85ddb776c9fc4d8292c92eaf88bc43a5/roles/a7d20885b6d94928a34b3cc5b3a00006 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:15:27.002752 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2c6434-fdda-410e-a326-7ecc5f689bba None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:27.003382 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2c6434-fdda-410e-a326-7ecc5f689bba None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:27.003856 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2c6434-fdda-410e-a326-7ecc5f689bba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:27.008319 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 934/1879] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Jul 25 12:15:26 2022] PUT /identity/v3/projects/3d38f98edb0a486199cae6655eeb3ed6/users/2ca0b9452f774b01a727942017e25628/roles/a7d20885b6d94928a34b3cc5b3a00006 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:15:27.019469 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a96ea15-36c4-4916-8617-7228c786bd8e None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:27.019795 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a96ea15-36c4-4916-8617-7228c786bd8e None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:27.020453 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a96ea15-36c4-4916-8617-7228c786bd8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:27.038974 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-6a2c6434-fdda-410e-a326-7ecc5f689bba None None] MFA Rules not processed for user `85ddb776c9fc4d8292c92eaf88bc43a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:15:27.040282 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-6a2c6434-fdda-410e-a326-7ecc5f689bba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:15:27.047402 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:15:27.050949 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-2a96ea15-36c4-4916-8617-7228c786bd8e None None] MFA Rules not processed for user `2ca0b9452f774b01a727942017e25628`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:15:27.054549 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-2a96ea15-36c4-4916-8617-7228c786bd8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:15:27.062516 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:15:27.139591 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 946/1881] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:15:27 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:15:27.139822 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 935/1881] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:15:27 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 12:15:27.149626 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2c6434-fdda-410e-a326-7ecc5f689bba None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:27.150985 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-6a2c6434-fdda-410e-a326-7ecc5f689bba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:15:27.158009 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a96ea15-36c4-4916-8617-7228c786bd8e None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:27.159189 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-2a96ea15-36c4-4916-8617-7228c786bd8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:15:27.222523 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a96ea15-36c4-4916-8617-7228c786bd8e None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:27.228192 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2c6434-fdda-410e-a326-7ecc5f689bba None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:27.228672 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614f52e8-3663-42da-9d8b-a039d1cf8d9e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f89ce6f0cf04373a414d927fa0f8a7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:27.229338 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614f52e8-3663-42da-9d8b-a039d1cf8d9e service cinder] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:27.230431 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614f52e8-3663-42da-9d8b-a039d1cf8d9e service cinder] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:27.230733 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614f52e8-3663-42da-9d8b-a039d1cf8d9e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:27.232741 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d1054a-afcb-460f-b5ae-30eb47806811 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f89ce6f0cf04373a414d927fa0f8a7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:27.233613 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d1054a-afcb-460f-b5ae-30eb47806811 service cinder] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:27.234002 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d1054a-afcb-460f-b5ae-30eb47806811 service cinder] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:27.234383 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d1054a-afcb-460f-b5ae-30eb47806811 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:27.242497 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614f52e8-3663-42da-9d8b-a039d1cf8d9e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:27.244005 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614f52e8-3663-42da-9d8b-a039d1cf8d9e service cinder] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:27.254975 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d1054a-afcb-460f-b5ae-30eb47806811 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:27.256717 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d1054a-afcb-460f-b5ae-30eb47806811 service cinder] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:27.260359 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 936/1882] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:15:27 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 12:15:27.270385 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 947/1883] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:15:27 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 12:15:27.431714 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614f52e8-3663-42da-9d8b-a039d1cf8d9e service cinder] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:27.431984 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614f52e8-3663-42da-9d8b-a039d1cf8d9e service cinder] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:27.441925 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c673b9-8b40-4f34-bff6-b273b6db3c35 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f89ce6f0cf04373a414d927fa0f8a7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:27.442658 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c673b9-8b40-4f34-bff6-b273b6db3c35 service cinder] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:27.442857 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c673b9-8b40-4f34-bff6-b273b6db3c35 service cinder] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:27.443146 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c673b9-8b40-4f34-bff6-b273b6db3c35 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:27.445213 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-28c673b9-8b40-4f34-bff6-b273b6db3c35 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:15:27.516503 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c673b9-8b40-4f34-bff6-b273b6db3c35 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:27.531833 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c673b9-8b40-4f34-bff6-b273b6db3c35 service cinder] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:27.559350 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 937/1884] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:15:27 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:15:28.335798 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d1054a-afcb-460f-b5ae-30eb47806811 service cinder] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:28.406491 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c673b9-8b40-4f34-bff6-b273b6db3c35 service cinder] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:28.422246 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d1054a-afcb-460f-b5ae-30eb47806811 service cinder] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:28.429323 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5179c06-a660-490e-8366-e2d4fd60c9a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:28.430510 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5179c06-a660-490e-8366-e2d4fd60c9a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:28.430998 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5179c06-a660-490e-8366-e2d4fd60c9a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:28.431819 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5179c06-a660-490e-8366-e2d4fd60c9a0 admin admin] PATH_INFO: `/v3/users/b2fe4b0d2f024c16b3a2aa0884453ea9` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:28.433472 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5179c06-a660-490e-8366-e2d4fd60c9a0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b2fe4b0d2f024c16b3a2aa0884453ea9)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:28.436093 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5179c06-a660-490e-8366-e2d4fd60c9a0 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:28.522071 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:15:28.533523 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-d5179c06-a660-490e-8366-e2d4fd60c9a0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b2fe4b0d2f024c16b3a2aa0884453ea9'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:28.535573 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-d5179c06-a660-490e-8366-e2d4fd60c9a0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b2fe4b0d2f024c16b3a2aa0884453ea9'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:28.552042 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-d5179c06-a660-490e-8366-e2d4fd60c9a0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b2fe4b0d2f024c16b3a2aa0884453ea9'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:28.564904 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:15:28.578611 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c673b9-8b40-4f34-bff6-b273b6db3c35 service cinder] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:28.583878 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-d5179c06-a660-490e-8366-e2d4fd60c9a0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b2fe4b0d2f024c16b3a2aa0884453ea9'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:28.588154 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ab8bfb-c29b-439e-86b5-9818f09744fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:28.589579 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ab8bfb-c29b-439e-86b5-9818f09744fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:28.590813 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ab8bfb-c29b-439e-86b5-9818f09744fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:28.591302 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ab8bfb-c29b-439e-86b5-9818f09744fa admin admin] PATH_INFO: `/v3/users/f7049d550b83487ea633f231e986cd66` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:28.593002 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ab8bfb-c29b-439e-86b5-9818f09744fa admin admin] RBAC: Authorizing `identity:delete_user(user_id=f7049d550b83487ea633f231e986cd66)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:28.596719 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ab8bfb-c29b-439e-86b5-9818f09744fa admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:28.601175 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-d5179c06-a660-490e-8366-e2d4fd60c9a0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b2fe4b0d2f024c16b3a2aa0884453ea9'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:28.617562 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 948/1885] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Jul 25 12:15:28 2022] DELETE /identity/v3/users/b2fe4b0d2f024c16b3a2aa0884453ea9 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 12:15:28.622302 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e0d6c6-4570-4eef-b621-e57db9ad9a56 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:28.622506 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e0d6c6-4570-4eef-b621-e57db9ad9a56 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:28.622780 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e0d6c6-4570-4eef-b621-e57db9ad9a56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:28.671568 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-b9e0d6c6-4570-4eef-b621-e57db9ad9a56 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:15:28.673142 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-b9e0d6c6-4570-4eef-b621-e57db9ad9a56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:15:28.689774 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-a4ab8bfb-c29b-439e-86b5-9818f09744fa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f7049d550b83487ea633f231e986cd66'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:28.709540 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-a4ab8bfb-c29b-439e-86b5-9818f09744fa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f7049d550b83487ea633f231e986cd66'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:28.744023 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-a4ab8bfb-c29b-439e-86b5-9818f09744fa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f7049d550b83487ea633f231e986cd66'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:28.787724 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-a4ab8bfb-c29b-439e-86b5-9818f09744fa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f7049d550b83487ea633f231e986cd66'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:28.789011 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-a4ab8bfb-c29b-439e-86b5-9818f09744fa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f7049d550b83487ea633f231e986cd66'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:28.798440 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 938/1886] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Jul 25 12:15:28 2022] DELETE /identity/v3/users/f7049d550b83487ea633f231e986cd66 => generated 0 bytes in 393 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:15:28.801886 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffd18ae-1e25-470e-8bfb-fe131099c46c None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:28.801886 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffd18ae-1e25-470e-8bfb-fe131099c46c None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:28.801886 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffd18ae-1e25-470e-8bfb-fe131099c46c None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:28.802510 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 939/1887] 10.222.0.42 () {54 vars in 921 bytes} [Mon Jul 25 12:15:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:15:28.804546 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fc9766-a5b1-4ed0-aeef-fbb30fcabb7b None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:28.804968 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fc9766-a5b1-4ed0-aeef-fbb30fcabb7b None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:28.805315 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fc9766-a5b1-4ed0-aeef-fbb30fcabb7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:28.810501 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 949/1888] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:15:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 12:15:28.821075 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e0d6c6-4570-4eef-b621-e57db9ad9a56 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:28.822744 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-b9e0d6c6-4570-4eef-b621-e57db9ad9a56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:15:28.835369 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-61fc9766-a5b1-4ed0-aeef-fbb30fcabb7b None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:15:28.837218 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-61fc9766-a5b1-4ed0-aeef-fbb30fcabb7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:15:28.921255 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e0d6c6-4570-4eef-b621-e57db9ad9a56 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:28.979156 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 940/1889] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:15:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:15:28.984721 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ab291c-d755-49e6-b37d-1e7c9aa10ad5 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:28.985018 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ab291c-d755-49e6-b37d-1e7c9aa10ad5 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:28.986379 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ab291c-d755-49e6-b37d-1e7c9aa10ad5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:28.996800 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cd9e63-f62c-4a75-8931-1d8e499fb18b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51733781201243e8abf9eca39b207750', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:28.998062 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cd9e63-f62c-4a75-8931-1d8e499fb18b service neutron] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:28.998563 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cd9e63-f62c-4a75-8931-1d8e499fb18b service neutron] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:28.999158 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cd9e63-f62c-4a75-8931-1d8e499fb18b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:29.014478 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8cd9e63-f62c-4a75-8931-1d8e499fb18b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:29.016513 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8cd9e63-f62c-4a75-8931-1d8e499fb18b service neutron] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:29.034095 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-33ab291c-d755-49e6-b37d-1e7c9aa10ad5 None None] MFA Rules not processed for user `f67142a1193d47c7bd33d7040268233c`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:15:29.035624 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-33ab291c-d755-49e6-b37d-1e7c9aa10ad5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:15:29.038505 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 950/1890] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:15:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:15:29.039525 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cd9e63-f62c-4a75-8931-1d8e499fb18b service neutron] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:29.052361 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cd9e63-f62c-4a75-8931-1d8e499fb18b service neutron] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:29.059502 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bbdf30-f8b2-43ac-8946-b73c21d75b62 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51733781201243e8abf9eca39b207750', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:29.060592 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bbdf30-f8b2-43ac-8946-b73c21d75b62 service neutron] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:29.061024 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bbdf30-f8b2-43ac-8946-b73c21d75b62 service neutron] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:29.061469 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bbdf30-f8b2-43ac-8946-b73c21d75b62 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:29.076746 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bbdf30-f8b2-43ac-8946-b73c21d75b62 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:29.079078 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bbdf30-f8b2-43ac-8946-b73c21d75b62 service neutron] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:29.095409 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 951/1891] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:15:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:15:29.108192 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:15:29.143859 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 941/1892] 10.222.0.42 () {58 vars in 1018 bytes} [Mon Jul 25 12:15:28 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:15:29.167922 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bbdf30-f8b2-43ac-8946-b73c21d75b62 service neutron] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:29.170407 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-87bbdf30-f8b2-43ac-8946-b73c21d75b62 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:15:29.195946 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bbdf30-f8b2-43ac-8946-b73c21d75b62 service neutron] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:29.202031 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9b7d71-7ef3-4bfb-b011-09877fae17a9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f67142a1193d47c7bd33d7040268233c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:29.202831 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9b7d71-7ef3-4bfb-b011-09877fae17a9 service nova] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:29.203125 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9b7d71-7ef3-4bfb-b011-09877fae17a9 service nova] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:29.203575 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9b7d71-7ef3-4bfb-b011-09877fae17a9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:29.215585 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9b7d71-7ef3-4bfb-b011-09877fae17a9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:29.217336 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9b7d71-7ef3-4bfb-b011-09877fae17a9 service nova] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:29.234281 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 952/1893] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:15:29 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:15:29.698077 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ab291c-d755-49e6-b37d-1e7c9aa10ad5 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:29.700172 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-33ab291c-d755-49e6-b37d-1e7c9aa10ad5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:15:29.715078 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ab291c-d755-49e6-b37d-1e7c9aa10ad5 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:29.720553 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3771ebf0-9c6b-4562-a969-fb267283d1b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:29.721794 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3771ebf0-9c6b-4562-a969-fb267283d1b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:29.722286 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3771ebf0-9c6b-4562-a969-fb267283d1b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:29.722767 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3771ebf0-9c6b-4562-a969-fb267283d1b7 admin admin] PATH_INFO: `/v3/projects/c86a86619d764987bbb889ca894dbc50` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:29.724384 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3771ebf0-9c6b-4562-a969-fb267283d1b7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c86a86619d764987bbb889ca894dbc50)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:29.726622 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3771ebf0-9c6b-4562-a969-fb267283d1b7 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:29.820259 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-3771ebf0-9c6b-4562-a969-fb267283d1b7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c86a86619d764987bbb889ca894dbc50', 'request_id': 'req-3771ebf0-9c6b-4562-a969-fb267283d1b7'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:29.830446 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-3771ebf0-9c6b-4562-a969-fb267283d1b7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c86a86619d764987bbb889ca894dbc50', 'request_id': 'req-3771ebf0-9c6b-4562-a969-fb267283d1b7'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:29.837945 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-3771ebf0-9c6b-4562-a969-fb267283d1b7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c86a86619d764987bbb889ca894dbc50', 'request_id': 'req-3771ebf0-9c6b-4562-a969-fb267283d1b7'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:29.857276 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-3771ebf0-9c6b-4562-a969-fb267283d1b7 admin admin] The token cache is being invalidate because project c86a86619d764987bbb889ca894dbc50 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 12:15:29.857821 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-3771ebf0-9c6b-4562-a969-fb267283d1b7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:29.858387 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-3771ebf0-9c6b-4562-a969-fb267283d1b7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:29.862724 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 942/1894] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Jul 25 12:15:29 2022] DELETE /identity/v3/projects/c86a86619d764987bbb889ca894dbc50 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:15:29.867514 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9b7d71-7ef3-4bfb-b011-09877fae17a9 service nova] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:29.895149 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9b7d71-7ef3-4bfb-b011-09877fae17a9 service nova] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:29.910305 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:15:30.010388 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3771ebf0-9c6b-4562-a969-fb267283d1b7 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:30.012305 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-3771ebf0-9c6b-4562-a969-fb267283d1b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:15:30.051529 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9d6f06-5a5d-49dd-8fdc-579ba790c86a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:30.052480 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9d6f06-5a5d-49dd-8fdc-579ba790c86a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:30.052779 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9d6f06-5a5d-49dd-8fdc-579ba790c86a admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:30.053276 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9d6f06-5a5d-49dd-8fdc-579ba790c86a admin admin] PATH_INFO: `/v3/users/85ddb776c9fc4d8292c92eaf88bc43a5` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:30.055640 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9d6f06-5a5d-49dd-8fdc-579ba790c86a admin admin] RBAC: Authorizing `identity:delete_user(user_id=85ddb776c9fc4d8292c92eaf88bc43a5)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:30.058104 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9d6f06-5a5d-49dd-8fdc-579ba790c86a admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:30.146326 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-0e9d6f06-5a5d-49dd-8fdc-579ba790c86a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '85ddb776c9fc4d8292c92eaf88bc43a5'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:30.147894 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-0e9d6f06-5a5d-49dd-8fdc-579ba790c86a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '85ddb776c9fc4d8292c92eaf88bc43a5'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:30.171791 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3771ebf0-9c6b-4562-a969-fb267283d1b7 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:30.172351 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-0e9d6f06-5a5d-49dd-8fdc-579ba790c86a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '85ddb776c9fc4d8292c92eaf88bc43a5'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:30.189172 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3f3f57-3520-4444-aa85-d4e4ad5e8d1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:30.191822 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3f3f57-3520-4444-aa85-d4e4ad5e8d1f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:30.192839 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3f3f57-3520-4444-aa85-d4e4ad5e8d1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:30.193717 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3f3f57-3520-4444-aa85-d4e4ad5e8d1f admin admin] PATH_INFO: `/v3/projects/c9c48b754f9341e0b2c415623ccdf9b6` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:30.198104 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3f3f57-3520-4444-aa85-d4e4ad5e8d1f admin admin] RBAC: Authorizing `identity:delete_project(project_id=c9c48b754f9341e0b2c415623ccdf9b6)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:30.200457 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3f3f57-3520-4444-aa85-d4e4ad5e8d1f admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:30.214073 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-0e9d6f06-5a5d-49dd-8fdc-579ba790c86a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '85ddb776c9fc4d8292c92eaf88bc43a5'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:30.229627 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-0e9d6f06-5a5d-49dd-8fdc-579ba790c86a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '85ddb776c9fc4d8292c92eaf88bc43a5'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:30.243499 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 953/1895] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Jul 25 12:15:29 2022] DELETE /identity/v3/users/85ddb776c9fc4d8292c92eaf88bc43a5 => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:15:30.284712 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-1a3f3f57-3520-4444-aa85-d4e4ad5e8d1f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c9c48b754f9341e0b2c415623ccdf9b6', 'request_id': 'req-1a3f3f57-3520-4444-aa85-d4e4ad5e8d1f'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:30.295105 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-1a3f3f57-3520-4444-aa85-d4e4ad5e8d1f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c9c48b754f9341e0b2c415623ccdf9b6', 'request_id': 'req-1a3f3f57-3520-4444-aa85-d4e4ad5e8d1f'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:30.316046 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-1a3f3f57-3520-4444-aa85-d4e4ad5e8d1f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c9c48b754f9341e0b2c415623ccdf9b6', 'request_id': 'req-1a3f3f57-3520-4444-aa85-d4e4ad5e8d1f'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:30.330882 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-1a3f3f57-3520-4444-aa85-d4e4ad5e8d1f admin admin] The token cache is being invalidate because project c9c48b754f9341e0b2c415623ccdf9b6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 12:15:30.331451 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-1a3f3f57-3520-4444-aa85-d4e4ad5e8d1f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:30.332224 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-1a3f3f57-3520-4444-aa85-d4e4ad5e8d1f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:30.337461 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 943/1896] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Jul 25 12:15:30 2022] DELETE /identity/v3/projects/c9c48b754f9341e0b2c415623ccdf9b6 => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:15:30.338616 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9d6f06-5a5d-49dd-8fdc-579ba790c86a admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:30.342217 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-0e9d6f06-5a5d-49dd-8fdc-579ba790c86a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:15:30.492980 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3f3f57-3520-4444-aa85-d4e4ad5e8d1f admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:30.493357 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9d6f06-5a5d-49dd-8fdc-579ba790c86a admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:30.494667 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-1a3f3f57-3520-4444-aa85-d4e4ad5e8d1f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:15:30.503703 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:15:30.548317 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:15:30.574719 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0119085-2ab6-4234-8fbc-c7d782c6fd95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:30.577119 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0119085-2ab6-4234-8fbc-c7d782c6fd95 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:30.577627 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0119085-2ab6-4234-8fbc-c7d782c6fd95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:30.578808 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0119085-2ab6-4234-8fbc-c7d782c6fd95 admin admin] PATH_INFO: `/v3/users/2ca0b9452f774b01a727942017e25628` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:30.580365 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0119085-2ab6-4234-8fbc-c7d782c6fd95 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2ca0b9452f774b01a727942017e25628)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:30.584031 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0119085-2ab6-4234-8fbc-c7d782c6fd95 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:30.587308 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3f3f57-3520-4444-aa85-d4e4ad5e8d1f admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:30.593486 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9d2989-9681-4917-9506-371a94625ee4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:30.595017 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9d2989-9681-4917-9506-371a94625ee4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:30.595820 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9d2989-9681-4917-9506-371a94625ee4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:30.596197 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9d2989-9681-4917-9506-371a94625ee4 admin admin] PATH_INFO: `/v3/users/4c4161f950274f50893d17f13778556c` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:30.598455 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9d2989-9681-4917-9506-371a94625ee4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4c4161f950274f50893d17f13778556c)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:30.601302 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9d2989-9681-4917-9506-371a94625ee4 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:30.653709 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-a0119085-2ab6-4234-8fbc-c7d782c6fd95 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2ca0b9452f774b01a727942017e25628'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:30.654515 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-a0119085-2ab6-4234-8fbc-c7d782c6fd95 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2ca0b9452f774b01a727942017e25628'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:30.666081 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-a0119085-2ab6-4234-8fbc-c7d782c6fd95 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2ca0b9452f774b01a727942017e25628'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:30.684311 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-a0119085-2ab6-4234-8fbc-c7d782c6fd95 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2ca0b9452f774b01a727942017e25628'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:30.693084 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-a0119085-2ab6-4234-8fbc-c7d782c6fd95 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2ca0b9452f774b01a727942017e25628'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:30.705481 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 954/1897] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Jul 25 12:15:30 2022] DELETE /identity/v3/users/2ca0b9452f774b01a727942017e25628 => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:15:30.716998 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-cf9d2989-9681-4917-9506-371a94625ee4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4c4161f950274f50893d17f13778556c'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:30.727411 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-cf9d2989-9681-4917-9506-371a94625ee4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4c4161f950274f50893d17f13778556c'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:30.740485 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-cf9d2989-9681-4917-9506-371a94625ee4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4c4161f950274f50893d17f13778556c'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:30.779259 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-cf9d2989-9681-4917-9506-371a94625ee4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4c4161f950274f50893d17f13778556c'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:30.782505 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-cf9d2989-9681-4917-9506-371a94625ee4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4c4161f950274f50893d17f13778556c'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:30.794258 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 944/1898] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Jul 25 12:15:30 2022] DELETE /identity/v3/users/4c4161f950274f50893d17f13778556c => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:15:30.797961 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3922d17b-da99-4487-9450-057af8781645 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:30.798258 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3922d17b-da99-4487-9450-057af8781645 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:30.798611 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3922d17b-da99-4487-9450-057af8781645 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:30.833861 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-3922d17b-da99-4487-9450-057af8781645 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:15:30.835277 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-3922d17b-da99-4487-9450-057af8781645 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:15:30.916259 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9d2989-9681-4917-9506-371a94625ee4 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:30.929955 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-cf9d2989-9681-4917-9506-371a94625ee4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:15:30.946613 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 955/1899] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:15:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:15:30.947141 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9d2989-9681-4917-9506-371a94625ee4 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:30.952484 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fe38be-2c02-44f2-b55f-943c024e657d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:30.953378 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fe38be-2c02-44f2-b55f-943c024e657d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:30.953777 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fe38be-2c02-44f2-b55f-943c024e657d admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:30.954070 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fe38be-2c02-44f2-b55f-943c024e657d admin admin] PATH_INFO: `/v3/projects/fcb6e4f7ec084ec98c46d8641aecb597` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:30.955428 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fe38be-2c02-44f2-b55f-943c024e657d admin admin] RBAC: Authorizing `identity:delete_project(project_id=fcb6e4f7ec084ec98c46d8641aecb597)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:30.957506 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fe38be-2c02-44f2-b55f-943c024e657d admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:30.958785 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3922d17b-da99-4487-9450-057af8781645 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:30.960881 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-3922d17b-da99-4487-9450-057af8781645 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:15:31.022813 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-e7fe38be-2c02-44f2-b55f-943c024e657d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fcb6e4f7ec084ec98c46d8641aecb597', 'request_id': 'req-e7fe38be-2c02-44f2-b55f-943c024e657d'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:31.036096 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-e7fe38be-2c02-44f2-b55f-943c024e657d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fcb6e4f7ec084ec98c46d8641aecb597', 'request_id': 'req-e7fe38be-2c02-44f2-b55f-943c024e657d'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:31.049596 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-e7fe38be-2c02-44f2-b55f-943c024e657d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fcb6e4f7ec084ec98c46d8641aecb597', 'request_id': 'req-e7fe38be-2c02-44f2-b55f-943c024e657d'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:31.063854 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3922d17b-da99-4487-9450-057af8781645 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:31.065277 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-e7fe38be-2c02-44f2-b55f-943c024e657d admin admin] The token cache is being invalidate because project fcb6e4f7ec084ec98c46d8641aecb597 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 12:15:31.065670 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-e7fe38be-2c02-44f2-b55f-943c024e657d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:31.066448 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-e7fe38be-2c02-44f2-b55f-943c024e657d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:31.068482 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 945/1900] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Jul 25 12:15:30 2022] DELETE /identity/v3/projects/fcb6e4f7ec084ec98c46d8641aecb597 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:15:31.073925 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4e45f4-a1b4-4492-9c67-5abb7afc2888 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:31.073925 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4e45f4-a1b4-4492-9c67-5abb7afc2888 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:31.074121 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4e45f4-a1b4-4492-9c67-5abb7afc2888 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:31.083243 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:15:31.094953 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:15:31.113210 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-0d4e45f4-a1b4-4492-9c67-5abb7afc2888 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:15:31.114802 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-0d4e45f4-a1b4-4492-9c67-5abb7afc2888 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:15:31.141763 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c48d9a6-55fc-429c-aed3-e6eacf036f2e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51733781201243e8abf9eca39b207750', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:31.142449 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c48d9a6-55fc-429c-aed3-e6eacf036f2e service neutron] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:31.142843 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c48d9a6-55fc-429c-aed3-e6eacf036f2e service neutron] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:31.143125 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c48d9a6-55fc-429c-aed3-e6eacf036f2e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:31.145691 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-4c48d9a6-55fc-429c-aed3-e6eacf036f2e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:15:31.205608 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 946/1901] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:15:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 12:15:31.206400 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4e45f4-a1b4-4492-9c67-5abb7afc2888 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:31.208128 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-0d4e45f4-a1b4-4492-9c67-5abb7afc2888 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:15:31.219867 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c48d9a6-55fc-429c-aed3-e6eacf036f2e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:31.221361 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c48d9a6-55fc-429c-aed3-e6eacf036f2e service neutron] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:31.223769 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4e45f4-a1b4-4492-9c67-5abb7afc2888 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:31.230234 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5728aebe-ed82-45f5-a7b7-f3fa92a65604 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:31.231425 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5728aebe-ed82-45f5-a7b7-f3fa92a65604 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:31.231845 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5728aebe-ed82-45f5-a7b7-f3fa92a65604 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:31.232224 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5728aebe-ed82-45f5-a7b7-f3fa92a65604 admin admin] PATH_INFO: `/v3/projects/3d38f98edb0a486199cae6655eeb3ed6` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:31.233671 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 956/1902] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:15:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 275 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:15:31.233919 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5728aebe-ed82-45f5-a7b7-f3fa92a65604 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3d38f98edb0a486199cae6655eeb3ed6)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:31.234998 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4268106-5140-452d-b85d-1de4e3dfec8b None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:31.235289 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4268106-5140-452d-b85d-1de4e3dfec8b None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:31.235587 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4268106-5140-452d-b85d-1de4e3dfec8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:31.237153 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5728aebe-ed82-45f5-a7b7-f3fa92a65604 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:31.258168 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-d4268106-5140-452d-b85d-1de4e3dfec8b None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:15:31.259450 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-d4268106-5140-452d-b85d-1de4e3dfec8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:15:31.270810 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 957/1903] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:15:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:15:31.275341 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4268106-5140-452d-b85d-1de4e3dfec8b None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:31.285379 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4268106-5140-452d-b85d-1de4e3dfec8b None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:31.291382 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5dcb81c-d96a-4149-9454-1d23b2b5449a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:31.293459 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dcb81c-d96a-4149-9454-1d23b2b5449a admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:31.293907 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dcb81c-d96a-4149-9454-1d23b2b5449a admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:31.294299 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dcb81c-d96a-4149-9454-1d23b2b5449a admin admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:31.295969 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5dcb81c-d96a-4149-9454-1d23b2b5449a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:15:31.296350 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5dcb81c-d96a-4149-9454-1d23b2b5449a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:31.299046 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5dcb81c-d96a-4149-9454-1d23b2b5449a admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:31.318391 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 958/1904] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Jul 25 12:15:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 12:15:31.323025 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5dcb81c-d96a-4149-9454-1d23b2b5449a admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:31.338833 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5dcb81c-d96a-4149-9454-1d23b2b5449a admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:31.342918 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-5728aebe-ed82-45f5-a7b7-f3fa92a65604 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3d38f98edb0a486199cae6655eeb3ed6', 'request_id': 'req-5728aebe-ed82-45f5-a7b7-f3fa92a65604'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:31.359475 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-5728aebe-ed82-45f5-a7b7-f3fa92a65604 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3d38f98edb0a486199cae6655eeb3ed6', 'request_id': 'req-5728aebe-ed82-45f5-a7b7-f3fa92a65604'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:31.388275 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-5728aebe-ed82-45f5-a7b7-f3fa92a65604 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3d38f98edb0a486199cae6655eeb3ed6', 'request_id': 'req-5728aebe-ed82-45f5-a7b7-f3fa92a65604'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:31.421079 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-5728aebe-ed82-45f5-a7b7-f3fa92a65604 admin admin] The token cache is being invalidate because project 3d38f98edb0a486199cae6655eeb3ed6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 12:15:31.421613 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-5728aebe-ed82-45f5-a7b7-f3fa92a65604 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:31.422458 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-5728aebe-ed82-45f5-a7b7-f3fa92a65604 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:31.424317 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 947/1905] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Jul 25 12:15:31 2022] DELETE /identity/v3/projects/3d38f98edb0a486199cae6655eeb3ed6 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:15:31.429250 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296d2dcb-0148-4c47-9f28-d6680e6cbbf8 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:31.429552 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296d2dcb-0148-4c47-9f28-d6680e6cbbf8 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:31.429840 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296d2dcb-0148-4c47-9f28-d6680e6cbbf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:31.447352 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e317dc45-63a8-4348-99d7-c795ca54ef48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:31.448577 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e317dc45-63a8-4348-99d7-c795ca54ef48 admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:31.449073 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e317dc45-63a8-4348-99d7-c795ca54ef48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:31.449681 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e317dc45-63a8-4348-99d7-c795ca54ef48 admin admin] PATH_INFO: `/v3/projects` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:31.450640 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e317dc45-63a8-4348-99d7-c795ca54ef48 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:31.452524 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-296d2dcb-0148-4c47-9f28-d6680e6cbbf8 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:15:31.453376 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e317dc45-63a8-4348-99d7-c795ca54ef48 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:31.453812 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-296d2dcb-0148-4c47-9f28-d6680e6cbbf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:15:31.463384 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 948/1906] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:15:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:15:31.484312 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a1cdcf-8527-4ea1-a111-d0ec27bb29a1 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:31.486608 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a1cdcf-8527-4ea1-a111-d0ec27bb29a1 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:31.487017 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a1cdcf-8527-4ea1-a111-d0ec27bb29a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:31.499198 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 959/1907] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Jul 25 12:15:31 2022] POST /identity/v3/projects => generated 359 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 12:15:31.503683 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e317dc45-63a8-4348-99d7-c795ca54ef48 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:31.510566 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-e317dc45-63a8-4348-99d7-c795ca54ef48 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:15:31.530014 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-50a1cdcf-8527-4ea1-a111-d0ec27bb29a1 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:15:31.535329 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-50a1cdcf-8527-4ea1-a111-d0ec27bb29a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:15:31.620498 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:15:31.629318 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 949/1908] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:15:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 12:15:31.630810 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a1cdcf-8527-4ea1-a111-d0ec27bb29a1 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:31.646205 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a1cdcf-8527-4ea1-a111-d0ec27bb29a1 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:31.668650 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558c832b-f9b2-4f72-ba37-080f88fa2125 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:31.670292 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558c832b-f9b2-4f72-ba37-080f88fa2125 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:31.671093 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558c832b-f9b2-4f72-ba37-080f88fa2125 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:31.672378 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558c832b-f9b2-4f72-ba37-080f88fa2125 admin admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:31.673566 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558c832b-f9b2-4f72-ba37-080f88fa2125 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:15:31.675264 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558c832b-f9b2-4f72-ba37-080f88fa2125 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:31.679652 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558c832b-f9b2-4f72-ba37-080f88fa2125 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:31.697684 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e317dc45-63a8-4348-99d7-c795ca54ef48 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:31.707979 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f541288-0501-43af-9b7f-bf126db59ce7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:31.708690 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f541288-0501-43af-9b7f-bf126db59ce7 admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:31.708985 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f541288-0501-43af-9b7f-bf126db59ce7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:31.709272 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f541288-0501-43af-9b7f-bf126db59ce7 admin admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:31.710553 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f541288-0501-43af-9b7f-bf126db59ce7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:31.713354 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f541288-0501-43af-9b7f-bf126db59ce7 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:31.717675 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 950/1909] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Jul 25 12:15:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:15:31.719113 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558c832b-f9b2-4f72-ba37-080f88fa2125 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:31.751196 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 960/1910] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Jul 25 12:15:31 2022] POST /identity/v3/users => generated 280 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:15:31.751379 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558c832b-f9b2-4f72-ba37-080f88fa2125 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:31.752286 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f541288-0501-43af-9b7f-bf126db59ce7 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:31.763084 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b2afb8-d7bd-4c8c-bdc2-04d75254a9ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:31.765297 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b2afb8-d7bd-4c8c-bdc2-04d75254a9ab admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:31.765595 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b2afb8-d7bd-4c8c-bdc2-04d75254a9ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:31.766556 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b2afb8-d7bd-4c8c-bdc2-04d75254a9ab admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:31.768560 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f541288-0501-43af-9b7f-bf126db59ce7 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:31.769467 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b2afb8-d7bd-4c8c-bdc2-04d75254a9ab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:31.771468 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b2afb8-d7bd-4c8c-bdc2-04d75254a9ab admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:31.781384 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ae5be6-66de-440d-999a-4a9507c8bdb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:31.782087 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ae5be6-66de-440d-999a-4a9507c8bdb2 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:31.782382 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ae5be6-66de-440d-999a-4a9507c8bdb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:31.782671 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ae5be6-66de-440d-999a-4a9507c8bdb2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:31.783444 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ae5be6-66de-440d-999a-4a9507c8bdb2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:31.785418 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ae5be6-66de-440d-999a-4a9507c8bdb2 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:31.795819 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:15:31.801782 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 961/1911] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Jul 25 12:15:31 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:15:31.806049 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ae5be6-66de-440d-999a-4a9507c8bdb2 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:31.823075 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ae5be6-66de-440d-999a-4a9507c8bdb2 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:31.829713 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4636716e-bebf-48eb-8a00-909ff241c3d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:31.831721 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4636716e-bebf-48eb-8a00-909ff241c3d1 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:31.831721 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4636716e-bebf-48eb-8a00-909ff241c3d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:31.832196 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4636716e-bebf-48eb-8a00-909ff241c3d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:31.833444 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4636716e-bebf-48eb-8a00-909ff241c3d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:31.836335 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4636716e-bebf-48eb-8a00-909ff241c3d1 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:31.852240 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 962/1913] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Jul 25 12:15:31 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:15:31.855358 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 951/1913] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Jul 25 12:15:31 2022] POST /identity/v3/projects => generated 385 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 12:15:31.856268 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4636716e-bebf-48eb-8a00-909ff241c3d1 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:31.856379 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b2afb8-d7bd-4c8c-bdc2-04d75254a9ab admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:31.877756 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4636716e-bebf-48eb-8a00-909ff241c3d1 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:31.983746 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b2afb8-d7bd-4c8c-bdc2-04d75254a9ab admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:31.988093 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eab6815-22f3-439f-9e80-9df974c05b0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:31.988845 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eab6815-22f3-439f-9e80-9df974c05b0d admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:31.989884 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eab6815-22f3-439f-9e80-9df974c05b0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:31.990171 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eab6815-22f3-439f-9e80-9df974c05b0d admin admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:31.991696 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eab6815-22f3-439f-9e80-9df974c05b0d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:31.992898 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b35f195-793f-428f-8e5d-8cffa51cca62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:31.993809 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eab6815-22f3-439f-9e80-9df974c05b0d admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:31.997997 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b35f195-793f-428f-8e5d-8cffa51cca62 admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:31.998487 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b35f195-793f-428f-8e5d-8cffa51cca62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:31.999130 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b35f195-793f-428f-8e5d-8cffa51cca62 admin admin] PATH_INFO: `/v3/projects/2b5390e94d444bfe91467df0e3f09457/users/b9dda03d7acc4ca18904c499b5276965/roles/472313ae69174578824df0bed6390744` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:32.041458 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 963/1914] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Jul 25 12:15:31 2022] POST /identity/v3/users => generated 293 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:15:32.045480 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eab6815-22f3-439f-9e80-9df974c05b0d admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:32.052337 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b35f195-793f-428f-8e5d-8cffa51cca62 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2b5390e94d444bfe91467df0e3f09457, user_id=b9dda03d7acc4ca18904c499b5276965, role_id=472313ae69174578824df0bed6390744)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:32.067839 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b35f195-793f-428f-8e5d-8cffa51cca62 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:32.068286 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eab6815-22f3-439f-9e80-9df974c05b0d admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:32.074065 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21fbd2d-dd8e-44e1-8e93-fbe6e6f66b43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:32.074899 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21fbd2d-dd8e-44e1-8e93-fbe6e6f66b43 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:32.075228 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21fbd2d-dd8e-44e1-8e93-fbe6e6f66b43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:32.075630 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21fbd2d-dd8e-44e1-8e93-fbe6e6f66b43 admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:32.076439 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21fbd2d-dd8e-44e1-8e93-fbe6e6f66b43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:32.077991 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21fbd2d-dd8e-44e1-8e93-fbe6e6f66b43 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:32.088876 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 952/1915] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Jul 25 12:15:31 2022] PUT /identity/v3/projects/2b5390e94d444bfe91467df0e3f09457/users/b9dda03d7acc4ca18904c499b5276965/roles/472313ae69174578824df0bed6390744 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:15:32.089750 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b35f195-793f-428f-8e5d-8cffa51cca62 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:32.099484 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 964/1916] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Jul 25 12:15:32 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 12:15:32.101487 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb295a3-0d77-42b2-9eb6-b6e9c39c2d6a None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:32.101781 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb295a3-0d77-42b2-9eb6-b6e9c39c2d6a None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:32.102064 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb295a3-0d77-42b2-9eb6-b6e9c39c2d6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:32.113664 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-7b35f195-793f-428f-8e5d-8cffa51cca62 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:15:32.139751 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-8bb295a3-0d77-42b2-9eb6-b6e9c39c2d6a None None] MFA Rules not processed for user `b9dda03d7acc4ca18904c499b5276965`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:15:32.141813 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-8bb295a3-0d77-42b2-9eb6-b6e9c39c2d6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:15:32.210651 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b35f195-793f-428f-8e5d-8cffa51cca62 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:32.216545 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4206de-4eb5-442f-9683-3304a76448b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:32.217505 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4206de-4eb5-442f-9683-3304a76448b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:32.217895 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4206de-4eb5-442f-9683-3304a76448b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:32.218246 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4206de-4eb5-442f-9683-3304a76448b0 admin admin] PATH_INFO: `/v3/projects/d0ecb7d347874891b39b6519dd76d907` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:32.220333 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4206de-4eb5-442f-9683-3304a76448b0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d0ecb7d347874891b39b6519dd76d907)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:32.223052 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4206de-4eb5-442f-9683-3304a76448b0 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:32.229723 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 965/1917] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:15:32 2022] POST /identity/v3/auth/tokens => generated 3226 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:15:32.231070 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb295a3-0d77-42b2-9eb6-b6e9c39c2d6a None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:32.275731 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb295a3-0d77-42b2-9eb6-b6e9c39c2d6a None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:32.294593 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f361f4f-5a78-4a68-b3fb-b30399ab1afd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:32.295292 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f361f4f-5a78-4a68-b3fb-b30399ab1afd admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:32.295963 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f361f4f-5a78-4a68-b3fb-b30399ab1afd admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:32.295963 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f361f4f-5a78-4a68-b3fb-b30399ab1afd admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:32.296614 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f361f4f-5a78-4a68-b3fb-b30399ab1afd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:32.310330 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-0c4206de-4eb5-442f-9683-3304a76448b0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd0ecb7d347874891b39b6519dd76d907', 'request_id': 'req-0c4206de-4eb5-442f-9683-3304a76448b0'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:32.323004 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-0c4206de-4eb5-442f-9683-3304a76448b0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd0ecb7d347874891b39b6519dd76d907', 'request_id': 'req-0c4206de-4eb5-442f-9683-3304a76448b0'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:32.324152 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f361f4f-5a78-4a68-b3fb-b30399ab1afd admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:32.331626 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-0c4206de-4eb5-442f-9683-3304a76448b0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd0ecb7d347874891b39b6519dd76d907', 'request_id': 'req-0c4206de-4eb5-442f-9683-3304a76448b0'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:32.333134 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:15:32.349811 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-0c4206de-4eb5-442f-9683-3304a76448b0 admin admin] The token cache is being invalidate because project d0ecb7d347874891b39b6519dd76d907 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 12:15:32.350124 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-0c4206de-4eb5-442f-9683-3304a76448b0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:32.351094 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-0c4206de-4eb5-442f-9683-3304a76448b0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:32.357642 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 953/1918] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Jul 25 12:15:32 2022] DELETE /identity/v3/projects/d0ecb7d347874891b39b6519dd76d907 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:15:32.358645 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4206de-4eb5-442f-9683-3304a76448b0 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:32.361864 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 966/1919] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Jul 25 12:15:32 2022] GET /identity/v3/roles => generated 1420 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:15:32.362019 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-0c4206de-4eb5-442f-9683-3304a76448b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:15:32.362925 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f361f4f-5a78-4a68-b3fb-b30399ab1afd admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:32.369129 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-9f361f4f-5a78-4a68-b3fb-b30399ab1afd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:15:32.371162 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:15:32.386419 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f361f4f-5a78-4a68-b3fb-b30399ab1afd admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:32.446700 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4206de-4eb5-442f-9683-3304a76448b0 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:32.451283 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3096f50e-1163-49e7-8b12-d97f42db2b32 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f89ce6f0cf04373a414d927fa0f8a7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:32.452076 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3096f50e-1163-49e7-8b12-d97f42db2b32 service cinder] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:32.452571 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3096f50e-1163-49e7-8b12-d97f42db2b32 service cinder] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:32.452890 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3096f50e-1163-49e7-8b12-d97f42db2b32 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:32.456418 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-3096f50e-1163-49e7-8b12-d97f42db2b32 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:15:32.497777 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afbaba7-f335-49f2-ae29-88a331a053fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:32.498951 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afbaba7-f335-49f2-ae29-88a331a053fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:32.498951 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afbaba7-f335-49f2-ae29-88a331a053fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:32.499240 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afbaba7-f335-49f2-ae29-88a331a053fb admin admin] PATH_INFO: `/v3/users/bb97468743ab4d6cb3c476b7b521821d` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:32.500580 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afbaba7-f335-49f2-ae29-88a331a053fb admin admin] RBAC: Authorizing `identity:delete_user(user_id=bb97468743ab4d6cb3c476b7b521821d)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:32.502613 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afbaba7-f335-49f2-ae29-88a331a053fb admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:32.554583 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3096f50e-1163-49e7-8b12-d97f42db2b32 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:32.556848 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3096f50e-1163-49e7-8b12-d97f42db2b32 service cinder] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:32.568458 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 954/1920] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:15:32 2022] GET /identity/v3/auth/tokens => generated 3226 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:15:32.569455 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3096f50e-1163-49e7-8b12-d97f42db2b32 service cinder] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:32.570780 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-3096f50e-1163-49e7-8b12-d97f42db2b32 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:15:32.578973 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-4afbaba7-f335-49f2-ae29-88a331a053fb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bb97468743ab4d6cb3c476b7b521821d'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:32.579722 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-4afbaba7-f335-49f2-ae29-88a331a053fb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bb97468743ab4d6cb3c476b7b521821d'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:32.589714 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-4afbaba7-f335-49f2-ae29-88a331a053fb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bb97468743ab4d6cb3c476b7b521821d'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:32.593905 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3096f50e-1163-49e7-8b12-d97f42db2b32 service cinder] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:32.607631 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a059e4-975d-4696-9f24-53ec146cb534 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:32.607631 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a059e4-975d-4696-9f24-53ec146cb534 admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:32.607631 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a059e4-975d-4696-9f24-53ec146cb534 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:32.607631 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a059e4-975d-4696-9f24-53ec146cb534 admin admin] PATH_INFO: `/v3/projects/5b5f1647200d4b32bf1cb523c293c018/users/94005bee8d8e4d62aa9f5ec84e7d6726/roles/472313ae69174578824df0bed6390744` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:32.608507 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-4afbaba7-f335-49f2-ae29-88a331a053fb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bb97468743ab4d6cb3c476b7b521821d'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:32.617078 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-4afbaba7-f335-49f2-ae29-88a331a053fb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bb97468743ab4d6cb3c476b7b521821d'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:32.618911 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a059e4-975d-4696-9f24-53ec146cb534 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b5f1647200d4b32bf1cb523c293c018, user_id=94005bee8d8e4d62aa9f5ec84e7d6726, role_id=472313ae69174578824df0bed6390744)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:32.621829 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a059e4-975d-4696-9f24-53ec146cb534 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:32.627188 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 967/1921] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Jul 25 12:15:32 2022] DELETE /identity/v3/users/bb97468743ab4d6cb3c476b7b521821d => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:15:32.628197 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afbaba7-f335-49f2-ae29-88a331a053fb admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:32.629724 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-4afbaba7-f335-49f2-ae29-88a331a053fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:15:32.652285 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 955/1922] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Jul 25 12:15:32 2022] PUT /identity/v3/projects/5b5f1647200d4b32bf1cb523c293c018/users/94005bee8d8e4d62aa9f5ec84e7d6726/roles/472313ae69174578824df0bed6390744 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:15:32.664924 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31a39f6-5dae-41ad-bc75-bfdc09c55dd7 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:32.665221 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31a39f6-5dae-41ad-bc75-bfdc09c55dd7 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:32.665635 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31a39f6-5dae-41ad-bc75-bfdc09c55dd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:32.696858 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-e31a39f6-5dae-41ad-bc75-bfdc09c55dd7 None None] MFA Rules not processed for user `94005bee8d8e4d62aa9f5ec84e7d6726`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:15:32.698133 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-e31a39f6-5dae-41ad-bc75-bfdc09c55dd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:15:32.734859 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afbaba7-f335-49f2-ae29-88a331a053fb admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:32.761747 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:15:32.804483 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 956/1923] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:15:32 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 12:15:32.816670 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31a39f6-5dae-41ad-bc75-bfdc09c55dd7 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:32.817725 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ed48db-7e11-48f7-b9c2-58eaca9cab6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:32.818546 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-e31a39f6-5dae-41ad-bc75-bfdc09c55dd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:15:32.820609 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ed48db-7e11-48f7-b9c2-58eaca9cab6d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:32.821481 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ed48db-7e11-48f7-b9c2-58eaca9cab6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:32.822446 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ed48db-7e11-48f7-b9c2-58eaca9cab6d admin admin] PATH_INFO: `/v3/users/b9dda03d7acc4ca18904c499b5276965` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:32.824229 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ed48db-7e11-48f7-b9c2-58eaca9cab6d admin admin] RBAC: Authorizing `identity:delete_user(user_id=b9dda03d7acc4ca18904c499b5276965)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:32.826170 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ed48db-7e11-48f7-b9c2-58eaca9cab6d admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:32.858182 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:15:32.893586 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-f1ed48db-7e11-48f7-b9c2-58eaca9cab6d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b9dda03d7acc4ca18904c499b5276965'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:32.894493 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-f1ed48db-7e11-48f7-b9c2-58eaca9cab6d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b9dda03d7acc4ca18904c499b5276965'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:32.903882 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31a39f6-5dae-41ad-bc75-bfdc09c55dd7 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:32.908122 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-f1ed48db-7e11-48f7-b9c2-58eaca9cab6d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b9dda03d7acc4ca18904c499b5276965'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:32.914116 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea65124a-91d1-4c4a-9f37-1af0db0d5fac service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f89ce6f0cf04373a414d927fa0f8a7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:32.915661 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea65124a-91d1-4c4a-9f37-1af0db0d5fac service cinder] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:32.916100 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea65124a-91d1-4c4a-9f37-1af0db0d5fac service cinder] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:32.916649 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea65124a-91d1-4c4a-9f37-1af0db0d5fac service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:32.926454 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-f1ed48db-7e11-48f7-b9c2-58eaca9cab6d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b9dda03d7acc4ca18904c499b5276965'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:32.931483 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea65124a-91d1-4c4a-9f37-1af0db0d5fac service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:32.933670 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea65124a-91d1-4c4a-9f37-1af0db0d5fac service cinder] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:32.934579 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-f1ed48db-7e11-48f7-b9c2-58eaca9cab6d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b9dda03d7acc4ca18904c499b5276965'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:32.942605 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 968/1924] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Jul 25 12:15:32 2022] DELETE /identity/v3/users/b9dda03d7acc4ca18904c499b5276965 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:15:32.946770 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f464ec3c-62d0-4375-bd45-5cc8ab53a91b None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:32.947859 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f464ec3c-62d0-4375-bd45-5cc8ab53a91b None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:32.948150 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f464ec3c-62d0-4375-bd45-5cc8ab53a91b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:32.949993 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 957/1925] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:15:32 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:15:32.970180 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-f464ec3c-62d0-4375-bd45-5cc8ab53a91b None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:15:32.971384 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-f464ec3c-62d0-4375-bd45-5cc8ab53a91b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:15:32.973295 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea65124a-91d1-4c4a-9f37-1af0db0d5fac service cinder] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:32.974323 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-ea65124a-91d1-4c4a-9f37-1af0db0d5fac service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:15:32.989465 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea65124a-91d1-4c4a-9f37-1af0db0d5fac service cinder] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:33.048081 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 969/1926] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:15:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:15:33.058115 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f464ec3c-62d0-4375-bd45-5cc8ab53a91b None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:33.058984 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbc0a7e-0297-4e63-8cd9-4d8b66810b9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:33.059816 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-f464ec3c-62d0-4375-bd45-5cc8ab53a91b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:15:33.060057 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbc0a7e-0297-4e63-8cd9-4d8b66810b9a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:33.060696 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbc0a7e-0297-4e63-8cd9-4d8b66810b9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:33.060696 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbc0a7e-0297-4e63-8cd9-4d8b66810b9a admin admin] PATH_INFO: `/v3/users/94005bee8d8e4d62aa9f5ec84e7d6726` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:33.062092 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbc0a7e-0297-4e63-8cd9-4d8b66810b9a admin admin] RBAC: Authorizing `identity:delete_user(user_id=94005bee8d8e4d62aa9f5ec84e7d6726)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:33.064182 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbc0a7e-0297-4e63-8cd9-4d8b66810b9a admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:33.138028 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:15:33.159303 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f464ec3c-62d0-4375-bd45-5cc8ab53a91b None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:33.164918 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58c2765-7003-411e-a822-0e3ef18d1419 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51733781201243e8abf9eca39b207750', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:33.165971 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58c2765-7003-411e-a822-0e3ef18d1419 service neutron] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:33.166846 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58c2765-7003-411e-a822-0e3ef18d1419 service neutron] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:33.167227 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58c2765-7003-411e-a822-0e3ef18d1419 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:33.174500 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-1bbc0a7e-0297-4e63-8cd9-4d8b66810b9a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '94005bee8d8e4d62aa9f5ec84e7d6726'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:33.179367 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58c2765-7003-411e-a822-0e3ef18d1419 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:33.181434 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58c2765-7003-411e-a822-0e3ef18d1419 service neutron] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:33.192049 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-1bbc0a7e-0297-4e63-8cd9-4d8b66810b9a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '94005bee8d8e4d62aa9f5ec84e7d6726'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:33.195120 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 970/1927] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:15:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 12:15:33.197498 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58c2765-7003-411e-a822-0e3ef18d1419 service neutron] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:33.199415 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-c58c2765-7003-411e-a822-0e3ef18d1419 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:15:33.202332 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-1bbc0a7e-0297-4e63-8cd9-4d8b66810b9a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '94005bee8d8e4d62aa9f5ec84e7d6726'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:33.213614 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58c2765-7003-411e-a822-0e3ef18d1419 service neutron] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:33.218610 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c63d97e-2c9d-4db4-908c-ebe6e001bb7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:33.219478 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c63d97e-2c9d-4db4-908c-ebe6e001bb7e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:33.219811 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c63d97e-2c9d-4db4-908c-ebe6e001bb7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:33.220092 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c63d97e-2c9d-4db4-908c-ebe6e001bb7e admin admin] PATH_INFO: `/v3/projects/91029af1b4d24082992561293988d07c` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:33.221286 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c63d97e-2c9d-4db4-908c-ebe6e001bb7e admin admin] RBAC: Authorizing `identity:delete_project(project_id=91029af1b4d24082992561293988d07c)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:33.222309 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-1bbc0a7e-0297-4e63-8cd9-4d8b66810b9a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '94005bee8d8e4d62aa9f5ec84e7d6726'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:33.223188 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c63d97e-2c9d-4db4-908c-ebe6e001bb7e admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:33.223405 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-1bbc0a7e-0297-4e63-8cd9-4d8b66810b9a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '94005bee8d8e4d62aa9f5ec84e7d6726'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:33.232956 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 958/1928] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Jul 25 12:15:32 2022] DELETE /identity/v3/users/94005bee8d8e4d62aa9f5ec84e7d6726 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 12:15:33.238169 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f756dcd-1a48-4f2c-9bc8-7665e4ff7c00 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:33.238852 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f756dcd-1a48-4f2c-9bc8-7665e4ff7c00 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:33.239287 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f756dcd-1a48-4f2c-9bc8-7665e4ff7c00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:33.263633 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-0f756dcd-1a48-4f2c-9bc8-7665e4ff7c00 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:15:33.265821 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-0f756dcd-1a48-4f2c-9bc8-7665e4ff7c00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:15:33.279004 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-3c63d97e-2c9d-4db4-908c-ebe6e001bb7e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '91029af1b4d24082992561293988d07c', 'request_id': 'req-3c63d97e-2c9d-4db4-908c-ebe6e001bb7e'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:33.297963 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-3c63d97e-2c9d-4db4-908c-ebe6e001bb7e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '91029af1b4d24082992561293988d07c', 'request_id': 'req-3c63d97e-2c9d-4db4-908c-ebe6e001bb7e'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:33.306345 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-3c63d97e-2c9d-4db4-908c-ebe6e001bb7e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '91029af1b4d24082992561293988d07c', 'request_id': 'req-3c63d97e-2c9d-4db4-908c-ebe6e001bb7e'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:33.317928 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:15:33.319885 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-3c63d97e-2c9d-4db4-908c-ebe6e001bb7e admin admin] The token cache is being invalidate because project 91029af1b4d24082992561293988d07c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80298) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 12:15:33.320180 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-3c63d97e-2c9d-4db4-908c-ebe6e001bb7e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:33.321314 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-3c63d97e-2c9d-4db4-908c-ebe6e001bb7e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:33.323391 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 971/1929] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Jul 25 12:15:33 2022] DELETE /identity/v3/projects/91029af1b4d24082992561293988d07c => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:15:33.327673 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c63d97e-2c9d-4db4-908c-ebe6e001bb7e admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:33.329682 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-3c63d97e-2c9d-4db4-908c-ebe6e001bb7e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:15:33.342931 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 959/1930] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:15:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:15:33.354252 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f756dcd-1a48-4f2c-9bc8-7665e4ff7c00 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:33.355833 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-0f756dcd-1a48-4f2c-9bc8-7665e4ff7c00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:15:33.407723 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c63d97e-2c9d-4db4-908c-ebe6e001bb7e admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:33.413478 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38040a55-b365-42d7-af22-bc44be319433 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:33.414418 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38040a55-b365-42d7-af22-bc44be319433 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:33.414714 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38040a55-b365-42d7-af22-bc44be319433 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:33.415082 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38040a55-b365-42d7-af22-bc44be319433 admin admin] PATH_INFO: `/v3/users/4d28296ffc8b44839d2b225db131a80c` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:33.416579 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38040a55-b365-42d7-af22-bc44be319433 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4d28296ffc8b44839d2b225db131a80c)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:33.418624 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38040a55-b365-42d7-af22-bc44be319433 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:33.442643 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f756dcd-1a48-4f2c-9bc8-7665e4ff7c00 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:33.448264 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951e1cd8-6778-4c4b-b3b7-0e4a69d427e9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51733781201243e8abf9eca39b207750', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:33.448962 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951e1cd8-6778-4c4b-b3b7-0e4a69d427e9 service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:33.450066 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951e1cd8-6778-4c4b-b3b7-0e4a69d427e9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:33.450502 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951e1cd8-6778-4c4b-b3b7-0e4a69d427e9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:33.462448 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951e1cd8-6778-4c4b-b3b7-0e4a69d427e9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:33.464125 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951e1cd8-6778-4c4b-b3b7-0e4a69d427e9 service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:33.474563 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-38040a55-b365-42d7-af22-bc44be319433 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4d28296ffc8b44839d2b225db131a80c'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:33.475266 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-38040a55-b365-42d7-af22-bc44be319433 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4d28296ffc8b44839d2b225db131a80c'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:33.475503 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 960/1931] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:15:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 12:15:33.484100 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-38040a55-b365-42d7-af22-bc44be319433 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4d28296ffc8b44839d2b225db131a80c'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:33.496517 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:15:33.497680 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-38040a55-b365-42d7-af22-bc44be319433 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4d28296ffc8b44839d2b225db131a80c'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:33.505738 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-38040a55-b365-42d7-af22-bc44be319433 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4d28296ffc8b44839d2b225db131a80c'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:33.512686 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 972/1932] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Jul 25 12:15:33 2022] DELETE /identity/v3/users/4d28296ffc8b44839d2b225db131a80c => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:15:33.594159 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951e1cd8-6778-4c4b-b3b7-0e4a69d427e9 service neutron] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:33.596277 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-951e1cd8-6778-4c4b-b3b7-0e4a69d427e9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:15:33.633346 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951e1cd8-6778-4c4b-b3b7-0e4a69d427e9 service neutron] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:33.739199 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8227cd4-fdc3-497c-a71e-385a80f88afa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:33.740182 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8227cd4-fdc3-497c-a71e-385a80f88afa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:33.740543 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8227cd4-fdc3-497c-a71e-385a80f88afa admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:33.741415 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8227cd4-fdc3-497c-a71e-385a80f88afa admin admin] PATH_INFO: `/v3/projects/2b5390e94d444bfe91467df0e3f09457` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:33.744188 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8227cd4-fdc3-497c-a71e-385a80f88afa admin admin] RBAC: Authorizing `identity:delete_project(project_id=2b5390e94d444bfe91467df0e3f09457)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:33.748114 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8227cd4-fdc3-497c-a71e-385a80f88afa admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:33.838332 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-f8227cd4-fdc3-497c-a71e-385a80f88afa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2b5390e94d444bfe91467df0e3f09457', 'request_id': 'req-f8227cd4-fdc3-497c-a71e-385a80f88afa'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:33.848600 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-f8227cd4-fdc3-497c-a71e-385a80f88afa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2b5390e94d444bfe91467df0e3f09457', 'request_id': 'req-f8227cd4-fdc3-497c-a71e-385a80f88afa'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:33.855424 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-f8227cd4-fdc3-497c-a71e-385a80f88afa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2b5390e94d444bfe91467df0e3f09457', 'request_id': 'req-f8227cd4-fdc3-497c-a71e-385a80f88afa'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:33.888693 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-f8227cd4-fdc3-497c-a71e-385a80f88afa admin admin] The token cache is being invalidate because project 2b5390e94d444bfe91467df0e3f09457 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 12:15:33.888995 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-f8227cd4-fdc3-497c-a71e-385a80f88afa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:33.896434 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-f8227cd4-fdc3-497c-a71e-385a80f88afa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:33.898645 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 961/1933] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Jul 25 12:15:33 2022] DELETE /identity/v3/projects/2b5390e94d444bfe91467df0e3f09457 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:15:33.900222 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38040a55-b365-42d7-af22-bc44be319433 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:33.904223 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-38040a55-b365-42d7-af22-bc44be319433 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:15:33.907881 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9dee4a-8ffb-4c7f-9387-f0dd2e12bd08 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:33.909504 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9dee4a-8ffb-4c7f-9387-f0dd2e12bd08 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:33.910662 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9dee4a-8ffb-4c7f-9387-f0dd2e12bd08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:33.922616 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38040a55-b365-42d7-af22-bc44be319433 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:33.957541 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-ec9dee4a-8ffb-4c7f-9387-f0dd2e12bd08 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:15:33.958957 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-ec9dee4a-8ffb-4c7f-9387-f0dd2e12bd08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:15:33.992444 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:15:34.026535 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3c60a5-5d36-4a69-9ab0-e29a9b35912b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:34.028255 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3c60a5-5d36-4a69-9ab0-e29a9b35912b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:34.028725 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3c60a5-5d36-4a69-9ab0-e29a9b35912b admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:34.029161 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3c60a5-5d36-4a69-9ab0-e29a9b35912b admin admin] PATH_INFO: `/v3/projects/5b5f1647200d4b32bf1cb523c293c018` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:34.031100 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3c60a5-5d36-4a69-9ab0-e29a9b35912b admin admin] RBAC: Authorizing `identity:delete_project(project_id=5b5f1647200d4b32bf1cb523c293c018)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:34.033399 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3c60a5-5d36-4a69-9ab0-e29a9b35912b admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:34.054052 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 962/1934] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:15:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:15:34.054982 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9dee4a-8ffb-4c7f-9387-f0dd2e12bd08 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:34.057126 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-ec9dee4a-8ffb-4c7f-9387-f0dd2e12bd08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:15:34.070396 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9dee4a-8ffb-4c7f-9387-f0dd2e12bd08 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:34.075856 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8953efdd-01c6-44b1-9ad3-331bf64e2591 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:34.077600 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8953efdd-01c6-44b1-9ad3-331bf64e2591 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:34.077947 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8953efdd-01c6-44b1-9ad3-331bf64e2591 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:34.078223 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8953efdd-01c6-44b1-9ad3-331bf64e2591 admin admin] PATH_INFO: `/v3/projects/767d37cf832442adb1838fe183f42b65` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:34.079483 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8953efdd-01c6-44b1-9ad3-331bf64e2591 admin admin] RBAC: Authorizing `identity:delete_project(project_id=767d37cf832442adb1838fe183f42b65)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:34.081748 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8953efdd-01c6-44b1-9ad3-331bf64e2591 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:34.100730 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-fd3c60a5-5d36-4a69-9ab0-e29a9b35912b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5b5f1647200d4b32bf1cb523c293c018', 'request_id': 'req-fd3c60a5-5d36-4a69-9ab0-e29a9b35912b'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:34.109755 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-fd3c60a5-5d36-4a69-9ab0-e29a9b35912b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5b5f1647200d4b32bf1cb523c293c018', 'request_id': 'req-fd3c60a5-5d36-4a69-9ab0-e29a9b35912b'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:34.116702 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-fd3c60a5-5d36-4a69-9ab0-e29a9b35912b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5b5f1647200d4b32bf1cb523c293c018', 'request_id': 'req-fd3c60a5-5d36-4a69-9ab0-e29a9b35912b'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:34.127884 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-fd3c60a5-5d36-4a69-9ab0-e29a9b35912b admin admin] The token cache is being invalidate because project 5b5f1647200d4b32bf1cb523c293c018 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80298) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 12:15:34.128171 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-fd3c60a5-5d36-4a69-9ab0-e29a9b35912b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:34.128801 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-fd3c60a5-5d36-4a69-9ab0-e29a9b35912b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:34.130343 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 973/1935] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Jul 25 12:15:33 2022] DELETE /identity/v3/projects/5b5f1647200d4b32bf1cb523c293c018 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:15:34.131603 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606b2c13-0976-4937-8384-fd0c6258f352 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:34.131898 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606b2c13-0976-4937-8384-fd0c6258f352 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:34.132176 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606b2c13-0976-4937-8384-fd0c6258f352 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:34.133646 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-8953efdd-01c6-44b1-9ad3-331bf64e2591 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '767d37cf832442adb1838fe183f42b65', 'request_id': 'req-8953efdd-01c6-44b1-9ad3-331bf64e2591'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:34.142460 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-8953efdd-01c6-44b1-9ad3-331bf64e2591 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '767d37cf832442adb1838fe183f42b65', 'request_id': 'req-8953efdd-01c6-44b1-9ad3-331bf64e2591'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:34.148707 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-8953efdd-01c6-44b1-9ad3-331bf64e2591 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '767d37cf832442adb1838fe183f42b65', 'request_id': 'req-8953efdd-01c6-44b1-9ad3-331bf64e2591'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:34.154685 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-606b2c13-0976-4937-8384-fd0c6258f352 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:15:34.155927 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-606b2c13-0976-4937-8384-fd0c6258f352 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:15:34.163449 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-8953efdd-01c6-44b1-9ad3-331bf64e2591 admin admin] The token cache is being invalidate because project 767d37cf832442adb1838fe183f42b65 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 12:15:34.163877 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-8953efdd-01c6-44b1-9ad3-331bf64e2591 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:34.164916 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-8953efdd-01c6-44b1-9ad3-331bf64e2591 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:34.167723 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 963/1936] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Jul 25 12:15:34 2022] DELETE /identity/v3/projects/767d37cf832442adb1838fe183f42b65 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:15:34.169025 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4409456f-5ff5-4ef7-a901-22a8ac41d790 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:34.169366 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4409456f-5ff5-4ef7-a901-22a8ac41d790 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:34.169650 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4409456f-5ff5-4ef7-a901-22a8ac41d790 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:34.192630 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-4409456f-5ff5-4ef7-a901-22a8ac41d790 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:15:34.193838 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-4409456f-5ff5-4ef7-a901-22a8ac41d790 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:15:34.242196 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 974/1937] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:15:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:15:34.248248 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9770cdc7-262a-4b35-a0ac-9b5047ee86c5 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:34.248742 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9770cdc7-262a-4b35-a0ac-9b5047ee86c5 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:34.249251 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9770cdc7-262a-4b35-a0ac-9b5047ee86c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:34.262425 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 964/1938] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:15:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:15:34.263357 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4409456f-5ff5-4ef7-a901-22a8ac41d790 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:34.279715 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4409456f-5ff5-4ef7-a901-22a8ac41d790 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:34.283454 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-9770cdc7-262a-4b35-a0ac-9b5047ee86c5 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:15:34.285023 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-9770cdc7-262a-4b35-a0ac-9b5047ee86c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:15:34.286112 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1148fb4-d148-47d7-a3af-fc72da091776 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:34.286862 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1148fb4-d148-47d7-a3af-fc72da091776 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:34.287085 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1148fb4-d148-47d7-a3af-fc72da091776 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:34.287366 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1148fb4-d148-47d7-a3af-fc72da091776 admin admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:34.288823 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1148fb4-d148-47d7-a3af-fc72da091776 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:15:34.289141 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1148fb4-d148-47d7-a3af-fc72da091776 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:34.291223 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1148fb4-d148-47d7-a3af-fc72da091776 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:34.299696 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 975/1939] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:15:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:15:34.301841 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7e4c52-7414-4902-8d3c-1a193daac57c None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:34.302350 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7e4c52-7414-4902-8d3c-1a193daac57c None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:34.302797 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7e4c52-7414-4902-8d3c-1a193daac57c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:34.308268 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 965/1940] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Jul 25 12:15:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:15:34.308952 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34a1a99-1972-4b45-89b7-1ab0e3f8b3c6 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:34.309237 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34a1a99-1972-4b45-89b7-1ab0e3f8b3c6 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:34.309518 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34a1a99-1972-4b45-89b7-1ab0e3f8b3c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:34.332244 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-a34a1a99-1972-4b45-89b7-1ab0e3f8b3c6 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:15:34.333459 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-a34a1a99-1972-4b45-89b7-1ab0e3f8b3c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:15:34.340784 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-6c7e4c52-7414-4902-8d3c-1a193daac57c None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:15:34.342236 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-6c7e4c52-7414-4902-8d3c-1a193daac57c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:15:34.347419 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 966/1941] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:15:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:15:34.348412 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34a1a99-1972-4b45-89b7-1ab0e3f8b3c6 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:34.361873 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34a1a99-1972-4b45-89b7-1ab0e3f8b3c6 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:34.368836 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 976/1942] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:15:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:15:34.369822 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7e4c52-7414-4902-8d3c-1a193daac57c None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:34.376726 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79456e8-91ba-4cc3-b4eb-8785628509bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:34.378393 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79456e8-91ba-4cc3-b4eb-8785628509bf admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:34.378875 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79456e8-91ba-4cc3-b4eb-8785628509bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:34.380295 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:15:34.382151 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79456e8-91ba-4cc3-b4eb-8785628509bf admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:34.383174 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79456e8-91ba-4cc3-b4eb-8785628509bf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:34.386552 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79456e8-91ba-4cc3-b4eb-8785628509bf admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:34.388877 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7e4c52-7414-4902-8d3c-1a193daac57c None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:34.405289 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3890d6-4c3b-42b0-a5db-148809080065 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:34.405974 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3890d6-4c3b-42b0-a5db-148809080065 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:34.406265 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3890d6-4c3b-42b0-a5db-148809080065 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:34.406552 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3890d6-4c3b-42b0-a5db-148809080065 admin admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:34.408156 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3890d6-4c3b-42b0-a5db-148809080065 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:15:34.408477 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3890d6-4c3b-42b0-a5db-148809080065 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:34.411128 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3890d6-4c3b-42b0-a5db-148809080065 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:34.419127 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:15:34.430660 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 977/1943] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Jul 25 12:15:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:15:34.430660 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3890d6-4c3b-42b0-a5db-148809080065 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:34.430942 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 967/1944] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Jul 25 12:15:34 2022] POST /identity/v3/projects => generated 369 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:15:34.432724 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79456e8-91ba-4cc3-b4eb-8785628509bf admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:34.444350 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3890d6-4c3b-42b0-a5db-148809080065 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:34.514569 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c698a14-71aa-47b5-85d6-df87c8b050b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:34.514804 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79456e8-91ba-4cc3-b4eb-8785628509bf admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:34.515242 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c698a14-71aa-47b5-85d6-df87c8b050b4 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:34.515534 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c698a14-71aa-47b5-85d6-df87c8b050b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:34.515860 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c698a14-71aa-47b5-85d6-df87c8b050b4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:34.516596 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c698a14-71aa-47b5-85d6-df87c8b050b4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:15:34.516912 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c698a14-71aa-47b5-85d6-df87c8b050b4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:34.518503 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c698a14-71aa-47b5-85d6-df87c8b050b4 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:34.520706 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3299664a-098e-4f1f-a996-e028bb75efa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:34.522799 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3299664a-098e-4f1f-a996-e028bb75efa9 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:34.523288 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3299664a-098e-4f1f-a996-e028bb75efa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:34.523954 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3299664a-098e-4f1f-a996-e028bb75efa9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:34.524527 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3299664a-098e-4f1f-a996-e028bb75efa9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:34.527189 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3299664a-098e-4f1f-a996-e028bb75efa9 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:34.534979 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 978/1945] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Jul 25 12:15:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:15:34.534979 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c698a14-71aa-47b5-85d6-df87c8b050b4 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:34.546841 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c698a14-71aa-47b5-85d6-df87c8b050b4 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:34.552292 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d703c47b-892f-4f2c-b029-ee1fc059ae7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:34.552964 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d703c47b-892f-4f2c-b029-ee1fc059ae7d admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:34.553276 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d703c47b-892f-4f2c-b029-ee1fc059ae7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:34.553570 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d703c47b-892f-4f2c-b029-ee1fc059ae7d admin admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:34.554294 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d703c47b-892f-4f2c-b029-ee1fc059ae7d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:34.556237 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d703c47b-892f-4f2c-b029-ee1fc059ae7d admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:34.559684 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 968/1946] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Jul 25 12:15:34 2022] POST /identity/v3/projects => generated 385 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:15:34.559684 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3299664a-098e-4f1f-a996-e028bb75efa9 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:34.578117 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 979/1947] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Jul 25 12:15:34 2022] POST /identity/v3/users => generated 285 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:15:34.578994 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d703c47b-892f-4f2c-b029-ee1fc059ae7d admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:34.592763 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d703c47b-892f-4f2c-b029-ee1fc059ae7d admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:34.622984 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3299664a-098e-4f1f-a996-e028bb75efa9 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:34.627650 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270576ae-4904-4e0c-b2c1-83239a7eb75e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:34.628309 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270576ae-4904-4e0c-b2c1-83239a7eb75e admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:34.628593 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270576ae-4904-4e0c-b2c1-83239a7eb75e admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:34.628871 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270576ae-4904-4e0c-b2c1-83239a7eb75e admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:34.629565 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270576ae-4904-4e0c-b2c1-83239a7eb75e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:34.631492 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270576ae-4904-4e0c-b2c1-83239a7eb75e admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:34.649982 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0a953d-b8f4-4f70-a03b-4c72b197c785 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:34.650753 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0a953d-b8f4-4f70-a03b-4c72b197c785 admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:34.651105 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0a953d-b8f4-4f70-a03b-4c72b197c785 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:34.651419 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0a953d-b8f4-4f70-a03b-4c72b197c785 admin admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:34.652145 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0a953d-b8f4-4f70-a03b-4c72b197c785 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:34.654125 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0a953d-b8f4-4f70-a03b-4c72b197c785 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:34.665703 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 969/1948] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Jul 25 12:15:34 2022] POST /identity/v3/projects => generated 375 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:15:34.665703 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270576ae-4904-4e0c-b2c1-83239a7eb75e admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:34.679455 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 980/1949] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Jul 25 12:15:34 2022] POST /identity/v3/users => generated 293 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:15:34.679455 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0a953d-b8f4-4f70-a03b-4c72b197c785 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:34.687152 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0a953d-b8f4-4f70-a03b-4c72b197c785 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:34.730027 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270576ae-4904-4e0c-b2c1-83239a7eb75e admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:34.734722 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e06663d-407d-48a3-ad2d-682d39ccb092 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:34.735394 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e06663d-407d-48a3-ad2d-682d39ccb092 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:34.735708 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e06663d-407d-48a3-ad2d-682d39ccb092 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:34.735990 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e06663d-407d-48a3-ad2d-682d39ccb092 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:34.736718 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e06663d-407d-48a3-ad2d-682d39ccb092 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:34.738325 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e06663d-407d-48a3-ad2d-682d39ccb092 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:34.739954 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:15:34.747130 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb69ed1b-3ef8-4dc5-a0e7-096c42e0a39e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:34.748002 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb69ed1b-3ef8-4dc5-a0e7-096c42e0a39e admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:34.748287 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb69ed1b-3ef8-4dc5-a0e7-096c42e0a39e admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:34.748567 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb69ed1b-3ef8-4dc5-a0e7-096c42e0a39e admin admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:34.749265 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb69ed1b-3ef8-4dc5-a0e7-096c42e0a39e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:34.749383 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 970/1950] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Jul 25 12:15:34 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:15:34.750311 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e06663d-407d-48a3-ad2d-682d39ccb092 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:34.751172 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb69ed1b-3ef8-4dc5-a0e7-096c42e0a39e admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:34.760517 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e06663d-407d-48a3-ad2d-682d39ccb092 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:34.768552 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9066ef6-8150-4166-8ef5-9605e11ae942 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:34.769464 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9066ef6-8150-4166-8ef5-9605e11ae942 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:34.769913 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9066ef6-8150-4166-8ef5-9605e11ae942 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:34.770343 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9066ef6-8150-4166-8ef5-9605e11ae942 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:34.771221 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9066ef6-8150-4166-8ef5-9605e11ae942 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:34.773693 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9066ef6-8150-4166-8ef5-9605e11ae942 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:34.775244 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:15:34.778634 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 981/1951] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Jul 25 12:15:34 2022] POST /identity/v3/users => generated 288 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:15:34.786639 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb69ed1b-3ef8-4dc5-a0e7-096c42e0a39e admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:34.787437 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 971/1952] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Jul 25 12:15:34 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:15:34.789063 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9066ef6-8150-4166-8ef5-9605e11ae942 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:34.800852 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9066ef6-8150-4166-8ef5-9605e11ae942 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:34.803474 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb69ed1b-3ef8-4dc5-a0e7-096c42e0a39e admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:34.805726 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7c2e72-775d-4edf-a01f-99c3d9389775 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:34.806398 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7c2e72-775d-4edf-a01f-99c3d9389775 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:34.806695 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7c2e72-775d-4edf-a01f-99c3d9389775 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:34.806976 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7c2e72-775d-4edf-a01f-99c3d9389775 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:34.807821 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7c2e72-775d-4edf-a01f-99c3d9389775 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:34.808269 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678d535a-d123-4838-b5f5-d9b582082582 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:34.808951 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678d535a-d123-4838-b5f5-d9b582082582 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:34.809241 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678d535a-d123-4838-b5f5-d9b582082582 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:34.809521 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678d535a-d123-4838-b5f5-d9b582082582 admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:34.809614 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7c2e72-775d-4edf-a01f-99c3d9389775 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:34.810248 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678d535a-d123-4838-b5f5-d9b582082582 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:34.811848 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678d535a-d123-4838-b5f5-d9b582082582 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:34.821805 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 972/1953] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Jul 25 12:15:34 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:15:34.822667 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 982/1954] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Jul 25 12:15:34 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:15:34.822783 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7c2e72-775d-4edf-a01f-99c3d9389775 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:34.825696 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678d535a-d123-4838-b5f5-d9b582082582 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:34.834350 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7c2e72-775d-4edf-a01f-99c3d9389775 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:34.836301 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678d535a-d123-4838-b5f5-d9b582082582 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:34.839876 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9551ead-e3b3-4c20-8df4-6a821615467b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:34.840539 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9551ead-e3b3-4c20-8df4-6a821615467b admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:34.840827 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9551ead-e3b3-4c20-8df4-6a821615467b admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:34.840977 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0add6cfa-7372-4279-a6c0-0574ed7f29f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:34.841106 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9551ead-e3b3-4c20-8df4-6a821615467b admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:34.841617 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0add6cfa-7372-4279-a6c0-0574ed7f29f2 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:34.841833 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9551ead-e3b3-4c20-8df4-6a821615467b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:34.841922 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0add6cfa-7372-4279-a6c0-0574ed7f29f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:34.842166 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0add6cfa-7372-4279-a6c0-0574ed7f29f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:34.842944 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0add6cfa-7372-4279-a6c0-0574ed7f29f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:34.843421 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9551ead-e3b3-4c20-8df4-6a821615467b admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:34.844479 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0add6cfa-7372-4279-a6c0-0574ed7f29f2 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:34.853668 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 973/1955] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Jul 25 12:15:34 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:15:34.858074 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 983/1956] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Jul 25 12:15:34 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:15:34.858074 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0add6cfa-7372-4279-a6c0-0574ed7f29f2 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:34.859881 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9551ead-e3b3-4c20-8df4-6a821615467b admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:34.869813 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9551ead-e3b3-4c20-8df4-6a821615467b admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:34.869962 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0add6cfa-7372-4279-a6c0-0574ed7f29f2 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:34.874561 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af53737a-d687-44dc-acd0-4da451c2ef2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:34.875449 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af53737a-d687-44dc-acd0-4da451c2ef2e admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:34.875765 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af53737a-d687-44dc-acd0-4da451c2ef2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:34.875932 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6161c8-e149-4c17-a8b9-6ceb824a8713 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:34.876056 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af53737a-d687-44dc-acd0-4da451c2ef2e admin admin] PATH_INFO: `/v3/projects/a1611654253d4caf8345e8ce946d948e/users/691b83d9af36496b9fa1f46ca111c327/roles/472313ae69174578824df0bed6390744` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:34.876862 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6161c8-e149-4c17-a8b9-6ceb824a8713 admin admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:34.877439 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6161c8-e149-4c17-a8b9-6ceb824a8713 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:34.877823 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6161c8-e149-4c17-a8b9-6ceb824a8713 admin admin] PATH_INFO: `/v3/projects/20443f3a0d0346c9be21304cd10335cd/users/01bf176bab8d447d822a30193741b3b5/roles/472313ae69174578824df0bed6390744` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:34.895517 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af53737a-d687-44dc-acd0-4da451c2ef2e admin admin] RBAC: Authorizing `identity:create_grant(project_id=a1611654253d4caf8345e8ce946d948e, user_id=691b83d9af36496b9fa1f46ca111c327, role_id=472313ae69174578824df0bed6390744)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:34.898922 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af53737a-d687-44dc-acd0-4da451c2ef2e admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:34.902428 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6161c8-e149-4c17-a8b9-6ceb824a8713 admin admin] RBAC: Authorizing `identity:create_grant(project_id=20443f3a0d0346c9be21304cd10335cd, user_id=01bf176bab8d447d822a30193741b3b5, role_id=472313ae69174578824df0bed6390744)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:34.905521 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6161c8-e149-4c17-a8b9-6ceb824a8713 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:34.917894 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 974/1957] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Jul 25 12:15:34 2022] PUT /identity/v3/projects/a1611654253d4caf8345e8ce946d948e/users/691b83d9af36496b9fa1f46ca111c327/roles/472313ae69174578824df0bed6390744 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:15:34.918681 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af53737a-d687-44dc-acd0-4da451c2ef2e admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:34.925395 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 984/1958] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Jul 25 12:15:34 2022] PUT /identity/v3/projects/20443f3a0d0346c9be21304cd10335cd/users/01bf176bab8d447d822a30193741b3b5/roles/472313ae69174578824df0bed6390744 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 12:15:34.928447 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd95685-2122-4369-a233-2c4b47791a25 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:34.929210 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd95685-2122-4369-a233-2c4b47791a25 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:34.929691 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd95685-2122-4369-a233-2c4b47791a25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:34.965831 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-0bd95685-2122-4369-a233-2c4b47791a25 None None] MFA Rules not processed for user `691b83d9af36496b9fa1f46ca111c327`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:15:34.967607 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-0bd95685-2122-4369-a233-2c4b47791a25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:15:34.991399 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af53737a-d687-44dc-acd0-4da451c2ef2e admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:35.058366 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 985/1959] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:15:34 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 12:15:35.058366 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f65ccb-2889-4eac-bbb2-a285461b7d0e None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:35.058366 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f65ccb-2889-4eac-bbb2-a285461b7d0e None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:35.058366 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f65ccb-2889-4eac-bbb2-a285461b7d0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:35.069656 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5082e60c-b998-4dad-9df1-9ac2673c299d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:35.070733 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5082e60c-b998-4dad-9df1-9ac2673c299d admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:35.071114 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5082e60c-b998-4dad-9df1-9ac2673c299d admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:35.071492 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5082e60c-b998-4dad-9df1-9ac2673c299d admin admin] PATH_INFO: `/v3/projects/e670a56a1bb54244b3132f472a03418e/users/877327a5ef78451782b13e054540640e/roles/472313ae69174578824df0bed6390744` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:35.088142 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5082e60c-b998-4dad-9df1-9ac2673c299d admin admin] RBAC: Authorizing `identity:create_grant(project_id=e670a56a1bb54244b3132f472a03418e, user_id=877327a5ef78451782b13e054540640e, role_id=472313ae69174578824df0bed6390744)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:35.090296 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-54f65ccb-2889-4eac-bbb2-a285461b7d0e None None] MFA Rules not processed for user `01bf176bab8d447d822a30193741b3b5`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:15:35.091185 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5082e60c-b998-4dad-9df1-9ac2673c299d admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:35.091521 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-54f65ccb-2889-4eac-bbb2-a285461b7d0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:15:35.108783 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 975/1960] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Jul 25 12:15:34 2022] PUT /identity/v3/projects/e670a56a1bb54244b3132f472a03418e/users/877327a5ef78451782b13e054540640e/roles/472313ae69174578824df0bed6390744 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:15:35.109455 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5082e60c-b998-4dad-9df1-9ac2673c299d admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:35.169532 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:15:35.196990 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5082e60c-b998-4dad-9df1-9ac2673c299d admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:35.198399 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 986/1961] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:15:35 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:15:35.202365 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec6deeb-40bb-4300-bab5-75cd1d36bc23 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:35.202365 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec6deeb-40bb-4300-bab5-75cd1d36bc23 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:35.202365 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec6deeb-40bb-4300-bab5-75cd1d36bc23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:35.205914 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20cf84f-eb48-4aa3-9549-48c33a86e437 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:35.207416 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20cf84f-eb48-4aa3-9549-48c33a86e437 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:35.207416 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20cf84f-eb48-4aa3-9549-48c33a86e437 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:35.210067 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20cf84f-eb48-4aa3-9549-48c33a86e437 admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:35.211014 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20cf84f-eb48-4aa3-9549-48c33a86e437 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:35.213476 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20cf84f-eb48-4aa3-9549-48c33a86e437 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:35.235108 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-4ec6deeb-40bb-4300-bab5-75cd1d36bc23 None None] MFA Rules not processed for user `877327a5ef78451782b13e054540640e`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:15:35.236555 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-4ec6deeb-40bb-4300-bab5-75cd1d36bc23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:15:35.255463 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:15:35.273675 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 976/1962] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Jul 25 12:15:35 2022] POST /identity/v3/projects => generated 375 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:15:35.275776 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20cf84f-eb48-4aa3-9549-48c33a86e437 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:35.381203 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20cf84f-eb48-4aa3-9549-48c33a86e437 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:35.381961 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 987/1963] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:15:35 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:15:35.384182 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec6deeb-40bb-4300-bab5-75cd1d36bc23 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:35.388223 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805c3074-469a-40df-ad28-62551b5ddbea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:35.389313 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805c3074-469a-40df-ad28-62551b5ddbea admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:35.389666 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805c3074-469a-40df-ad28-62551b5ddbea admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:35.389957 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805c3074-469a-40df-ad28-62551b5ddbea admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:35.390676 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805c3074-469a-40df-ad28-62551b5ddbea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:35.392665 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805c3074-469a-40df-ad28-62551b5ddbea admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:35.412751 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec6deeb-40bb-4300-bab5-75cd1d36bc23 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:35.422657 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7ca48c-5682-45cb-b015-80141541dbb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:35.423366 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7ca48c-5682-45cb-b015-80141541dbb9 admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:35.424691 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7ca48c-5682-45cb-b015-80141541dbb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:35.424998 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7ca48c-5682-45cb-b015-80141541dbb9 admin admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:35.425734 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7ca48c-5682-45cb-b015-80141541dbb9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:35.432950 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7ca48c-5682-45cb-b015-80141541dbb9 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:35.446527 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 977/1964] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Jul 25 12:15:35 2022] POST /identity/v3/projects => generated 383 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:15:35.446527 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805c3074-469a-40df-ad28-62551b5ddbea admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:35.447782 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-805c3074-469a-40df-ad28-62551b5ddbea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:15:35.469549 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 988/1965] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Jul 25 12:15:35 2022] POST /identity/v3/users => generated 294 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:15:35.470775 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7ca48c-5682-45cb-b015-80141541dbb9 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:35.495197 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7ca48c-5682-45cb-b015-80141541dbb9 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:35.572258 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805c3074-469a-40df-ad28-62551b5ddbea admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:35.576590 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b43d2a2-94fe-4465-8c53-6dd17b3fcbfa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f89ce6f0cf04373a414d927fa0f8a7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:35.586120 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b43d2a2-94fe-4465-8c53-6dd17b3fcbfa service cinder] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:35.586120 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b43d2a2-94fe-4465-8c53-6dd17b3fcbfa service cinder] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:35.586370 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b43d2a2-94fe-4465-8c53-6dd17b3fcbfa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:35.660801 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bb0fc9-f7db-441b-a533-e42ab4968c0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:35.661518 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bb0fc9-f7db-441b-a533-e42ab4968c0c admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:35.661858 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bb0fc9-f7db-441b-a533-e42ab4968c0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:35.662066 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bb0fc9-f7db-441b-a533-e42ab4968c0c admin admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:35.663312 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bb0fc9-f7db-441b-a533-e42ab4968c0c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:35.669964 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bb0fc9-f7db-441b-a533-e42ab4968c0c admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:35.721478 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b43d2a2-94fe-4465-8c53-6dd17b3fcbfa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:35.726037 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 989/1966] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Jul 25 12:15:35 2022] POST /identity/v3/users => generated 292 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:15:35.728405 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b43d2a2-94fe-4465-8c53-6dd17b3fcbfa service cinder] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:35.734917 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bb0fc9-f7db-441b-a533-e42ab4968c0c admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:35.742465 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 978/1967] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:15:35 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:15:35.743182 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b43d2a2-94fe-4465-8c53-6dd17b3fcbfa service cinder] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:35.751524 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:15:35.756848 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b43d2a2-94fe-4465-8c53-6dd17b3fcbfa service cinder] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:35.764900 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90845e6-2232-427a-a429-a8bbffd064a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:35.765819 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90845e6-2232-427a-a429-a8bbffd064a4 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:35.766254 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90845e6-2232-427a-a429-a8bbffd064a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:35.766631 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90845e6-2232-427a-a429-a8bbffd064a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:35.767284 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90845e6-2232-427a-a429-a8bbffd064a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:35.770747 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bb0fc9-f7db-441b-a533-e42ab4968c0c admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:35.781184 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90845e6-2232-427a-a429-a8bbffd064a4 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:35.794197 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601c14f4-372f-4e1d-987a-24f3a6522c4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:35.794892 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601c14f4-372f-4e1d-987a-24f3a6522c4e admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:35.795183 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601c14f4-372f-4e1d-987a-24f3a6522c4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:35.795467 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601c14f4-372f-4e1d-987a-24f3a6522c4e admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:35.796245 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601c14f4-372f-4e1d-987a-24f3a6522c4e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:35.797840 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601c14f4-372f-4e1d-987a-24f3a6522c4e admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:35.808895 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 979/1968] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Jul 25 12:15:35 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:15:35.812190 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90845e6-2232-427a-a429-a8bbffd064a4 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:35.817949 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 990/1969] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Jul 25 12:15:35 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:15:35.822251 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601c14f4-372f-4e1d-987a-24f3a6522c4e admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:35.827867 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90845e6-2232-427a-a429-a8bbffd064a4 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:35.835197 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601c14f4-372f-4e1d-987a-24f3a6522c4e admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:35.835749 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae25e23-5ed7-4f22-84b4-419e25a7e32e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:35.836612 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae25e23-5ed7-4f22-84b4-419e25a7e32e admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:35.836903 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae25e23-5ed7-4f22-84b4-419e25a7e32e admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:35.837236 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae25e23-5ed7-4f22-84b4-419e25a7e32e admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:35.837970 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae25e23-5ed7-4f22-84b4-419e25a7e32e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:35.839653 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae25e23-5ed7-4f22-84b4-419e25a7e32e admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:35.842610 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ae902f-92e0-4420-a527-de48ff57ac75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:35.843678 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ae902f-92e0-4420-a527-de48ff57ac75 admin admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:35.844031 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ae902f-92e0-4420-a527-de48ff57ac75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:35.844381 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ae902f-92e0-4420-a527-de48ff57ac75 admin admin] PATH_INFO: `/v3/projects/b7472f6c889a4f5d90ad3ea58f2321f3/users/4ed4674b41f24e1f9a3142907b0847b4/roles/a7d20885b6d94928a34b3cc5b3a00006` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:35.851199 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 980/1970] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Jul 25 12:15:35 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:15:35.855304 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae25e23-5ed7-4f22-84b4-419e25a7e32e admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:35.867058 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae25e23-5ed7-4f22-84b4-419e25a7e32e admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:35.869618 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ae902f-92e0-4420-a527-de48ff57ac75 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b7472f6c889a4f5d90ad3ea58f2321f3, user_id=4ed4674b41f24e1f9a3142907b0847b4, role_id=a7d20885b6d94928a34b3cc5b3a00006)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:35.872472 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e5dd9a-538f-4f01-9282-cba7fd16acc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:35.873513 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e5dd9a-538f-4f01-9282-cba7fd16acc2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:35.873736 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e5dd9a-538f-4f01-9282-cba7fd16acc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:35.874055 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ae902f-92e0-4420-a527-de48ff57ac75 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:35.874215 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e5dd9a-538f-4f01-9282-cba7fd16acc2 admin admin] PATH_INFO: `/v3/projects/c3bd8c0338434e6c8b64de1c09f78813/users/09fb6a302d0c4407b912d8d1985e63b4/roles/472313ae69174578824df0bed6390744` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:35.893296 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e5dd9a-538f-4f01-9282-cba7fd16acc2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c3bd8c0338434e6c8b64de1c09f78813, user_id=09fb6a302d0c4407b912d8d1985e63b4, role_id=472313ae69174578824df0bed6390744)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:35.896086 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 991/1971] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Jul 25 12:15:35 2022] PUT /identity/v3/projects/b7472f6c889a4f5d90ad3ea58f2321f3/users/4ed4674b41f24e1f9a3142907b0847b4/roles/a7d20885b6d94928a34b3cc5b3a00006 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:15:35.899886 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e5dd9a-538f-4f01-9282-cba7fd16acc2 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:35.900884 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ae902f-92e0-4420-a527-de48ff57ac75 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:35.918527 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 981/1972] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Jul 25 12:15:35 2022] PUT /identity/v3/projects/c3bd8c0338434e6c8b64de1c09f78813/users/09fb6a302d0c4407b912d8d1985e63b4/roles/472313ae69174578824df0bed6390744 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 12:15:35.928596 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ee0d18-57c1-4c9d-b360-63525921c047 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:35.928893 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ee0d18-57c1-4c9d-b360-63525921c047 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:35.929180 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ee0d18-57c1-4c9d-b360-63525921c047 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:35.959011 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-80ee0d18-57c1-4c9d-b360-63525921c047 None None] MFA Rules not processed for user `09fb6a302d0c4407b912d8d1985e63b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:15:35.961251 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-80ee0d18-57c1-4c9d-b360-63525921c047 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:15:35.971536 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:15:36.001816 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ae902f-92e0-4420-a527-de48ff57ac75 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:36.077223 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 982/1973] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:15:35 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 12:15:36.083371 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ee0d18-57c1-4c9d-b360-63525921c047 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:36.109855 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82358e5-44c6-4ba6-b9e7-fcbb5e2b51a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:36.111085 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82358e5-44c6-4ba6-b9e7-fcbb5e2b51a1 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:36.111615 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82358e5-44c6-4ba6-b9e7-fcbb5e2b51a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:36.112080 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82358e5-44c6-4ba6-b9e7-fcbb5e2b51a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:36.114241 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82358e5-44c6-4ba6-b9e7-fcbb5e2b51a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:36.116352 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82358e5-44c6-4ba6-b9e7-fcbb5e2b51a1 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:36.131443 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 992/1974] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Jul 25 12:15:35 2022] GET /identity/v3/roles => generated 1420 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:15:36.134968 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82358e5-44c6-4ba6-b9e7-fcbb5e2b51a1 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:36.165872 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82358e5-44c6-4ba6-b9e7-fcbb5e2b51a1 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:36.179049 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5dbc57-7a5b-41f7-8444-dd43be363a14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:36.180160 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5dbc57-7a5b-41f7-8444-dd43be363a14 admin admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:36.183131 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5dbc57-7a5b-41f7-8444-dd43be363a14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:36.183530 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5dbc57-7a5b-41f7-8444-dd43be363a14 admin admin] PATH_INFO: `/v3/projects/b7472f6c889a4f5d90ad3ea58f2321f3/users/4ed4674b41f24e1f9a3142907b0847b4/roles/a7d20885b6d94928a34b3cc5b3a00006` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:36.189522 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5dbc57-7a5b-41f7-8444-dd43be363a14 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b7472f6c889a4f5d90ad3ea58f2321f3, user_id=4ed4674b41f24e1f9a3142907b0847b4, role_id=a7d20885b6d94928a34b3cc5b3a00006)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:36.198636 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5dbc57-7a5b-41f7-8444-dd43be363a14 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:36.215940 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ee0d18-57c1-4c9d-b360-63525921c047 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:36.220175 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 993/1975] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Jul 25 12:15:36 2022] PUT /identity/v3/projects/b7472f6c889a4f5d90ad3ea58f2321f3/users/4ed4674b41f24e1f9a3142907b0847b4/roles/a7d20885b6d94928a34b3cc5b3a00006 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 12:15:36.230312 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e477ea-3956-4d5e-b7fa-b63e4af8d841 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:36.230873 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e477ea-3956-4d5e-b7fa-b63e4af8d841 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:36.231686 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e477ea-3956-4d5e-b7fa-b63e4af8d841 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:36.276372 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-e3e477ea-3956-4d5e-b7fa-b63e4af8d841 None None] MFA Rules not processed for user `4ed4674b41f24e1f9a3142907b0847b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:15:36.277843 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-e3e477ea-3956-4d5e-b7fa-b63e4af8d841 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:15:36.306296 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:15:36.325568 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9f5993-f98b-470c-96a7-bdf35d8c0b75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:36.326327 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9f5993-f98b-470c-96a7-bdf35d8c0b75 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:36.327657 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9f5993-f98b-470c-96a7-bdf35d8c0b75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:36.328016 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9f5993-f98b-470c-96a7-bdf35d8c0b75 admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:36.329334 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9f5993-f98b-470c-96a7-bdf35d8c0b75 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:36.332191 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9f5993-f98b-470c-96a7-bdf35d8c0b75 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:36.383123 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 983/1976] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Jul 25 12:15:36 2022] POST /identity/v3/projects => generated 383 bytes in 300 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:15:36.384698 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9f5993-f98b-470c-96a7-bdf35d8c0b75 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:36.422619 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 994/1977] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:15:36 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:15:36.435652 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e477ea-3956-4d5e-b7fa-b63e4af8d841 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:36.466620 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9f5993-f98b-470c-96a7-bdf35d8c0b75 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:36.472411 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4cba22-601d-47da-bd7b-6be79543a97d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:36.473287 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4cba22-601d-47da-bd7b-6be79543a97d admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:36.473578 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4cba22-601d-47da-bd7b-6be79543a97d admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:36.473860 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4cba22-601d-47da-bd7b-6be79543a97d admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:36.474950 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4cba22-601d-47da-bd7b-6be79543a97d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:36.477709 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4cba22-601d-47da-bd7b-6be79543a97d admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:36.511650 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 984/1978] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Jul 25 12:15:36 2022] POST /identity/v3/users => generated 298 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:15:36.517714 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4cba22-601d-47da-bd7b-6be79543a97d admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:36.531073 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4cba22-601d-47da-bd7b-6be79543a97d admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:36.538835 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70f29cc-8994-4e92-b4ca-89f3005eb357 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:36.539856 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70f29cc-8994-4e92-b4ca-89f3005eb357 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:36.540614 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70f29cc-8994-4e92-b4ca-89f3005eb357 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:36.540614 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70f29cc-8994-4e92-b4ca-89f3005eb357 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:36.541576 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70f29cc-8994-4e92-b4ca-89f3005eb357 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:36.543373 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70f29cc-8994-4e92-b4ca-89f3005eb357 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:36.545082 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e477ea-3956-4d5e-b7fa-b63e4af8d841 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:36.556971 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 985/1979] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Jul 25 12:15:36 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:15:36.562398 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70f29cc-8994-4e92-b4ca-89f3005eb357 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:36.567016 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:15:36.583232 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70f29cc-8994-4e92-b4ca-89f3005eb357 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:36.592782 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54af3d6d-efce-4b44-aa0a-bd14abf4e02c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:36.593972 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54af3d6d-efce-4b44-aa0a-bd14abf4e02c admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:36.594442 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54af3d6d-efce-4b44-aa0a-bd14abf4e02c admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:36.594932 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54af3d6d-efce-4b44-aa0a-bd14abf4e02c admin admin] PATH_INFO: `/v3/projects/711841df205b426ca2f42a7b7e9b77be/users/2f3c1eb09f864673a821efca2edf8c27/roles/a7d20885b6d94928a34b3cc5b3a00006` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:36.626353 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54af3d6d-efce-4b44-aa0a-bd14abf4e02c admin admin] RBAC: Authorizing `identity:create_grant(project_id=711841df205b426ca2f42a7b7e9b77be, user_id=2f3c1eb09f864673a821efca2edf8c27, role_id=a7d20885b6d94928a34b3cc5b3a00006)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:36.629631 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54af3d6d-efce-4b44-aa0a-bd14abf4e02c admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:36.632427 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d961b1bb-c5a5-4d7e-88c3-7cef90ed35d4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f89ce6f0cf04373a414d927fa0f8a7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:36.633628 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d961b1bb-c5a5-4d7e-88c3-7cef90ed35d4 service cinder] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:36.634629 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d961b1bb-c5a5-4d7e-88c3-7cef90ed35d4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:36.635281 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d961b1bb-c5a5-4d7e-88c3-7cef90ed35d4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:36.649697 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 986/1980] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Jul 25 12:15:36 2022] PUT /identity/v3/projects/711841df205b426ca2f42a7b7e9b77be/users/2f3c1eb09f864673a821efca2edf8c27/roles/a7d20885b6d94928a34b3cc5b3a00006 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:15:36.654075 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54af3d6d-efce-4b44-aa0a-bd14abf4e02c admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:36.722371 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d961b1bb-c5a5-4d7e-88c3-7cef90ed35d4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:36.724189 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d961b1bb-c5a5-4d7e-88c3-7cef90ed35d4 service cinder] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:36.739908 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 995/1981] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:15:36 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 12:15:36.761123 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54af3d6d-efce-4b44-aa0a-bd14abf4e02c admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:36.777033 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb351f29-a822-4b47-abf3-41346ab7ea90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:36.777727 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb351f29-a822-4b47-abf3-41346ab7ea90 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:36.777988 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb351f29-a822-4b47-abf3-41346ab7ea90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:36.778250 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb351f29-a822-4b47-abf3-41346ab7ea90 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:36.779017 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb351f29-a822-4b47-abf3-41346ab7ea90 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:36.780866 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb351f29-a822-4b47-abf3-41346ab7ea90 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:36.805455 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 987/1982] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Jul 25 12:15:36 2022] GET /identity/v3/roles => generated 1420 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 12:15:36.806551 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d961b1bb-c5a5-4d7e-88c3-7cef90ed35d4 service cinder] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:36.824646 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d961b1bb-c5a5-4d7e-88c3-7cef90ed35d4 service cinder] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:36.845222 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7405d137-6264-41be-a512-3210c24d8c4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:36.846636 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7405d137-6264-41be-a512-3210c24d8c4d admin admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:36.846636 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7405d137-6264-41be-a512-3210c24d8c4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:36.846841 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7405d137-6264-41be-a512-3210c24d8c4d admin admin] PATH_INFO: `/v3/projects/711841df205b426ca2f42a7b7e9b77be/users/2f3c1eb09f864673a821efca2edf8c27/roles/a7d20885b6d94928a34b3cc5b3a00006` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:36.849105 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7405d137-6264-41be-a512-3210c24d8c4d admin admin] RBAC: Authorizing `identity:create_grant(project_id=711841df205b426ca2f42a7b7e9b77be, user_id=2f3c1eb09f864673a821efca2edf8c27, role_id=a7d20885b6d94928a34b3cc5b3a00006)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:36.851950 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7405d137-6264-41be-a512-3210c24d8c4d admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:36.864056 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 996/1983] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Jul 25 12:15:36 2022] PUT /identity/v3/projects/711841df205b426ca2f42a7b7e9b77be/users/2f3c1eb09f864673a821efca2edf8c27/roles/a7d20885b6d94928a34b3cc5b3a00006 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:15:36.876003 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316f7454-6eca-445b-bdb1-c33aba9e2de2 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:36.877228 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316f7454-6eca-445b-bdb1-c33aba9e2de2 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:36.877746 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316f7454-6eca-445b-bdb1-c33aba9e2de2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:36.909195 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-316f7454-6eca-445b-bdb1-c33aba9e2de2 None None] MFA Rules not processed for user `2f3c1eb09f864673a821efca2edf8c27`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:15:36.910834 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-316f7454-6eca-445b-bdb1-c33aba9e2de2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:15:36.933814 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:15:37.089516 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 988/1984] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:15:36 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:15:37.130178 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7405d137-6264-41be-a512-3210c24d8c4d admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:37.245096 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7405d137-6264-41be-a512-3210c24d8c4d admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:37.253733 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9159153-7d86-441d-b088-03aa123eb96c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f89ce6f0cf04373a414d927fa0f8a7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:37.254722 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9159153-7d86-441d-b088-03aa123eb96c service cinder] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:37.255222 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9159153-7d86-441d-b088-03aa123eb96c service cinder] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:37.255755 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9159153-7d86-441d-b088-03aa123eb96c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:37.352224 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9159153-7d86-441d-b088-03aa123eb96c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:37.353677 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9159153-7d86-441d-b088-03aa123eb96c service cinder] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:37.372623 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 997/1985] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:15:37 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 243 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:15:39.125922 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316f7454-6eca-445b-bdb1-c33aba9e2de2 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:39.138977 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316f7454-6eca-445b-bdb1-c33aba9e2de2 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:39.142934 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169c5ca2-dd53-455c-a28c-ef973370eaf9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f89ce6f0cf04373a414d927fa0f8a7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:39.143723 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169c5ca2-dd53-455c-a28c-ef973370eaf9 service cinder] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:39.143903 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169c5ca2-dd53-455c-a28c-ef973370eaf9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:39.144198 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169c5ca2-dd53-455c-a28c-ef973370eaf9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:39.217136 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169c5ca2-dd53-455c-a28c-ef973370eaf9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:39.231578 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169c5ca2-dd53-455c-a28c-ef973370eaf9 service cinder] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:39.245764 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 989/1986] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:15:39 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:15:39.684650 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9159153-7d86-441d-b088-03aa123eb96c service cinder] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:39.715294 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:15:39.749473 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169c5ca2-dd53-455c-a28c-ef973370eaf9 service cinder] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:39.871196 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9159153-7d86-441d-b088-03aa123eb96c service cinder] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:39.893849 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbae66b-2c15-4113-9371-ee88cd8d39fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:39.894766 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbae66b-2c15-4113-9371-ee88cd8d39fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:39.895055 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbae66b-2c15-4113-9371-ee88cd8d39fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:39.895336 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbae66b-2c15-4113-9371-ee88cd8d39fc admin admin] PATH_INFO: `/v3/users/691b83d9af36496b9fa1f46ca111c327` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:39.896617 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cbae66b-2c15-4113-9371-ee88cd8d39fc admin admin] RBAC: Authorizing `identity:delete_user(user_id=691b83d9af36496b9fa1f46ca111c327)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:39.898622 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cbae66b-2c15-4113-9371-ee88cd8d39fc admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:39.912929 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169c5ca2-dd53-455c-a28c-ef973370eaf9 service cinder] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:39.927850 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7e5bca-5157-4fca-9fcd-f699176bb455 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:39.953136 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7e5bca-5157-4fca-9fcd-f699176bb455 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:39.953442 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7e5bca-5157-4fca-9fcd-f699176bb455 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:39.953809 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7e5bca-5157-4fca-9fcd-f699176bb455 admin admin] PATH_INFO: `/v3/users/877327a5ef78451782b13e054540640e` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:39.955223 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7e5bca-5157-4fca-9fcd-f699176bb455 admin admin] RBAC: Authorizing `identity:delete_user(user_id=877327a5ef78451782b13e054540640e)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:39.957208 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7e5bca-5157-4fca-9fcd-f699176bb455 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:40.006518 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-8cbae66b-2c15-4113-9371-ee88cd8d39fc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '691b83d9af36496b9fa1f46ca111c327'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:40.007909 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-8cbae66b-2c15-4113-9371-ee88cd8d39fc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '691b83d9af36496b9fa1f46ca111c327'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:40.021878 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-8cbae66b-2c15-4113-9371-ee88cd8d39fc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '691b83d9af36496b9fa1f46ca111c327'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:40.075429 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-8cbae66b-2c15-4113-9371-ee88cd8d39fc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '691b83d9af36496b9fa1f46ca111c327'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:40.088343 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-ca7e5bca-5157-4fca-9fcd-f699176bb455 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '877327a5ef78451782b13e054540640e'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:40.096928 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-ca7e5bca-5157-4fca-9fcd-f699176bb455 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '877327a5ef78451782b13e054540640e'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:40.098920 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:15:40.102802 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-8cbae66b-2c15-4113-9371-ee88cd8d39fc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '691b83d9af36496b9fa1f46ca111c327'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:40.114364 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-ca7e5bca-5157-4fca-9fcd-f699176bb455 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '877327a5ef78451782b13e054540640e'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:40.120869 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 998/1987] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Jul 25 12:15:39 2022] DELETE /identity/v3/users/691b83d9af36496b9fa1f46ca111c327 => generated 0 bytes in 434 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:15:40.120869 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbae66b-2c15-4113-9371-ee88cd8d39fc admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:40.120869 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-8cbae66b-2c15-4113-9371-ee88cd8d39fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:15:40.131362 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-ca7e5bca-5157-4fca-9fcd-f699176bb455 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '877327a5ef78451782b13e054540640e'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:40.132109 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-ca7e5bca-5157-4fca-9fcd-f699176bb455 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '877327a5ef78451782b13e054540640e'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:40.142074 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 990/1988] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Jul 25 12:15:39 2022] DELETE /identity/v3/users/877327a5ef78451782b13e054540640e => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:15:40.142074 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f9d3b9-52b6-46f4-9585-dc05f20b6b39 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:40.142074 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f9d3b9-52b6-46f4-9585-dc05f20b6b39 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:40.142074 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f9d3b9-52b6-46f4-9585-dc05f20b6b39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:40.168142 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-23f9d3b9-52b6-46f4-9585-dc05f20b6b39 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:15:40.169607 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-23f9d3b9-52b6-46f4-9585-dc05f20b6b39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:15:40.198975 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbae66b-2c15-4113-9371-ee88cd8d39fc admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:40.277390 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:15:40.284489 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 991/1989] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:15:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:15:40.286821 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5db767d-8aaa-4153-aff1-1a950a931640 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f89ce6f0cf04373a414d927fa0f8a7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:40.288263 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37048fde-0de4-477c-9a87-9dc2385063f0 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:40.288924 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5db767d-8aaa-4153-aff1-1a950a931640 service cinder] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:40.289445 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5db767d-8aaa-4153-aff1-1a950a931640 service cinder] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:40.289802 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5db767d-8aaa-4153-aff1-1a950a931640 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:40.293400 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-c5db767d-8aaa-4153-aff1-1a950a931640 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:15:40.297963 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37048fde-0de4-477c-9a87-9dc2385063f0 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:40.298556 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37048fde-0de4-477c-9a87-9dc2385063f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:40.325865 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-37048fde-0de4-477c-9a87-9dc2385063f0 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:15:40.327497 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-37048fde-0de4-477c-9a87-9dc2385063f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:15:40.338408 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 992/1990] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:15:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:15:40.339300 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37048fde-0de4-477c-9a87-9dc2385063f0 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:40.340606 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-37048fde-0de4-477c-9a87-9dc2385063f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:15:40.371511 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5db767d-8aaa-4153-aff1-1a950a931640 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:40.373259 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5db767d-8aaa-4153-aff1-1a950a931640 service cinder] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:40.384296 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 999/1991] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:15:40 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 265 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:15:40.385135 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5db767d-8aaa-4153-aff1-1a950a931640 service cinder] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:40.386474 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-c5db767d-8aaa-4153-aff1-1a950a931640 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:15:40.403409 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37048fde-0de4-477c-9a87-9dc2385063f0 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:40.409424 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece1568e-6e7c-4958-972b-4b65d9aae28f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51733781201243e8abf9eca39b207750', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:40.410340 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece1568e-6e7c-4958-972b-4b65d9aae28f service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:40.410624 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece1568e-6e7c-4958-972b-4b65d9aae28f service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:40.410902 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece1568e-6e7c-4958-972b-4b65d9aae28f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:40.424112 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece1568e-6e7c-4958-972b-4b65d9aae28f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:40.425866 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece1568e-6e7c-4958-972b-4b65d9aae28f service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:40.450562 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 993/1992] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:15:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:15:40.460997 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5db767d-8aaa-4153-aff1-1a950a931640 service cinder] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:40.465253 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feadf547-b36e-4f31-9ae9-818c8d05a7c9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51733781201243e8abf9eca39b207750', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:40.466051 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feadf547-b36e-4f31-9ae9-818c8d05a7c9 service neutron] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:40.466451 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feadf547-b36e-4f31-9ae9-818c8d05a7c9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:40.466834 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feadf547-b36e-4f31-9ae9-818c8d05a7c9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:40.478533 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feadf547-b36e-4f31-9ae9-818c8d05a7c9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:40.480563 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feadf547-b36e-4f31-9ae9-818c8d05a7c9 service neutron] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:40.512621 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1000/1993] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:15:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:15:41.017667 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece1568e-6e7c-4958-972b-4b65d9aae28f service neutron] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:41.019655 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-ece1568e-6e7c-4958-972b-4b65d9aae28f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:15:41.037613 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece1568e-6e7c-4958-972b-4b65d9aae28f service neutron] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:41.048816 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292007a0-565f-4752-a102-03762dc74349 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:41.050348 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292007a0-565f-4752-a102-03762dc74349 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:41.050996 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292007a0-565f-4752-a102-03762dc74349 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:41.063927 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292007a0-565f-4752-a102-03762dc74349 admin admin] PATH_INFO: `/v3/projects/a1611654253d4caf8345e8ce946d948e` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:41.067175 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292007a0-565f-4752-a102-03762dc74349 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a1611654253d4caf8345e8ce946d948e)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:41.070843 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292007a0-565f-4752-a102-03762dc74349 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:41.129495 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-292007a0-565f-4752-a102-03762dc74349 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a1611654253d4caf8345e8ce946d948e', 'request_id': 'req-292007a0-565f-4752-a102-03762dc74349'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:41.140242 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-292007a0-565f-4752-a102-03762dc74349 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a1611654253d4caf8345e8ce946d948e', 'request_id': 'req-292007a0-565f-4752-a102-03762dc74349'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:41.146892 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-292007a0-565f-4752-a102-03762dc74349 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a1611654253d4caf8345e8ce946d948e', 'request_id': 'req-292007a0-565f-4752-a102-03762dc74349'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:41.148052 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:INSERT=2,keystone:DELETE=10,keystone:SELECT=88 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:15:41.161514 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-292007a0-565f-4752-a102-03762dc74349 admin admin] The token cache is being invalidate because project a1611654253d4caf8345e8ce946d948e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 12:15:41.161800 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-292007a0-565f-4752-a102-03762dc74349 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:41.162497 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-292007a0-565f-4752-a102-03762dc74349 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:41.164117 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 994/1994] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Jul 25 12:15:41 2022] DELETE /identity/v3/projects/a1611654253d4caf8345e8ce946d948e => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:15:41.168761 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feadf547-b36e-4f31-9ae9-818c8d05a7c9 service neutron] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:41.180796 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feadf547-b36e-4f31-9ae9-818c8d05a7c9 service neutron] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:41.238951 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41b55e6-f69c-40a8-a48b-e8faca3e2e7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:41.240308 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41b55e6-f69c-40a8-a48b-e8faca3e2e7b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:41.241162 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41b55e6-f69c-40a8-a48b-e8faca3e2e7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:41.241512 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41b55e6-f69c-40a8-a48b-e8faca3e2e7b admin admin] PATH_INFO: `/v3/users/4ed4674b41f24e1f9a3142907b0847b4` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:41.242788 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41b55e6-f69c-40a8-a48b-e8faca3e2e7b admin admin] RBAC: Authorizing `identity:delete_user(user_id=4ed4674b41f24e1f9a3142907b0847b4)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:41.244896 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41b55e6-f69c-40a8-a48b-e8faca3e2e7b admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:41.289796 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:15:41.294343 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-c41b55e6-f69c-40a8-a48b-e8faca3e2e7b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4ed4674b41f24e1f9a3142907b0847b4'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:41.295289 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-c41b55e6-f69c-40a8-a48b-e8faca3e2e7b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4ed4674b41f24e1f9a3142907b0847b4'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:41.305779 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-c41b55e6-f69c-40a8-a48b-e8faca3e2e7b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4ed4674b41f24e1f9a3142907b0847b4'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:41.319317 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-c41b55e6-f69c-40a8-a48b-e8faca3e2e7b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4ed4674b41f24e1f9a3142907b0847b4'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:41.326296 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-c41b55e6-f69c-40a8-a48b-e8faca3e2e7b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4ed4674b41f24e1f9a3142907b0847b4'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:41.332413 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1001/1995] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Jul 25 12:15:41 2022] DELETE /identity/v3/users/4ed4674b41f24e1f9a3142907b0847b4 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:15:41.349844 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292007a0-565f-4752-a102-03762dc74349 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:41.351803 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-292007a0-565f-4752-a102-03762dc74349 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:15:41.367000 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292007a0-565f-4752-a102-03762dc74349 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:41.440059 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d8b134-3ff0-40d0-bf79-a3b3fdf7f22a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:41.441066 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d8b134-3ff0-40d0-bf79-a3b3fdf7f22a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:41.442102 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d8b134-3ff0-40d0-bf79-a3b3fdf7f22a admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:41.442102 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d8b134-3ff0-40d0-bf79-a3b3fdf7f22a admin admin] PATH_INFO: `/v3/projects/e670a56a1bb54244b3132f472a03418e` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:41.442847 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d8b134-3ff0-40d0-bf79-a3b3fdf7f22a admin admin] RBAC: Authorizing `identity:delete_project(project_id=e670a56a1bb54244b3132f472a03418e)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:41.449328 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d8b134-3ff0-40d0-bf79-a3b3fdf7f22a admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:41.507717 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-87d8b134-3ff0-40d0-bf79-a3b3fdf7f22a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e670a56a1bb54244b3132f472a03418e', 'request_id': 'req-87d8b134-3ff0-40d0-bf79-a3b3fdf7f22a'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:41.522259 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-87d8b134-3ff0-40d0-bf79-a3b3fdf7f22a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e670a56a1bb54244b3132f472a03418e', 'request_id': 'req-87d8b134-3ff0-40d0-bf79-a3b3fdf7f22a'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:41.529927 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-87d8b134-3ff0-40d0-bf79-a3b3fdf7f22a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e670a56a1bb54244b3132f472a03418e', 'request_id': 'req-87d8b134-3ff0-40d0-bf79-a3b3fdf7f22a'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:41.543911 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-87d8b134-3ff0-40d0-bf79-a3b3fdf7f22a admin admin] The token cache is being invalidate because project e670a56a1bb54244b3132f472a03418e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 12:15:41.544287 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-87d8b134-3ff0-40d0-bf79-a3b3fdf7f22a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:41.545511 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-87d8b134-3ff0-40d0-bf79-a3b3fdf7f22a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:41.547362 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 995/1996] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Jul 25 12:15:41 2022] DELETE /identity/v3/projects/e670a56a1bb54244b3132f472a03418e => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:15:41.554659 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33fd88a-f635-4555-85da-bbd5f415a9ae None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:41.555225 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33fd88a-f635-4555-85da-bbd5f415a9ae None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:41.555513 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33fd88a-f635-4555-85da-bbd5f415a9ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:41.581857 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-b33fd88a-f635-4555-85da-bbd5f415a9ae None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:15:41.584061 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-b33fd88a-f635-4555-85da-bbd5f415a9ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:15:41.663855 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d8b134-3ff0-40d0-bf79-a3b3fdf7f22a admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:41.667199 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-87d8b134-3ff0-40d0-bf79-a3b3fdf7f22a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:15:41.686358 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1002/1997] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:15:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:15:41.695517 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc5cc97-4ff7-44df-b2e3-0a8edaa260e4 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:41.695933 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc5cc97-4ff7-44df-b2e3-0a8edaa260e4 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:41.696297 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc5cc97-4ff7-44df-b2e3-0a8edaa260e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:41.744747 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-efc5cc97-4ff7-44df-b2e3-0a8edaa260e4 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:15:41.745687 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-efc5cc97-4ff7-44df-b2e3-0a8edaa260e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:15:41.761138 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1003/1998] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:15:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:15:41.762292 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc5cc97-4ff7-44df-b2e3-0a8edaa260e4 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:41.773819 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc5cc97-4ff7-44df-b2e3-0a8edaa260e4 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:41.782658 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33645a49-7ece-4bbc-a1ed-810e481b0ce2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:41.783378 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33645a49-7ece-4bbc-a1ed-810e481b0ce2 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:41.783724 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33645a49-7ece-4bbc-a1ed-810e481b0ce2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:41.784136 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33645a49-7ece-4bbc-a1ed-810e481b0ce2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:41.785388 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33645a49-7ece-4bbc-a1ed-810e481b0ce2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:15:41.786001 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33645a49-7ece-4bbc-a1ed-810e481b0ce2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:41.788577 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33645a49-7ece-4bbc-a1ed-810e481b0ce2 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:41.806783 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1004/1999] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Jul 25 12:15:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:15:41.809288 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33645a49-7ece-4bbc-a1ed-810e481b0ce2 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:41.839806 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33645a49-7ece-4bbc-a1ed-810e481b0ce2 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:41.839994 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d8b134-3ff0-40d0-bf79-a3b3fdf7f22a admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:41.845819 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03ff066-335e-4c4f-920f-f2172941ba61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:41.846610 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03ff066-335e-4c4f-920f-f2172941ba61 admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:41.847016 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03ff066-335e-4c4f-920f-f2172941ba61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:41.848113 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03ff066-335e-4c4f-920f-f2172941ba61 admin admin] PATH_INFO: `/v3/projects` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:41.848956 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03ff066-335e-4c4f-920f-f2172941ba61 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:41.851984 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03ff066-335e-4c4f-920f-f2172941ba61 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:41.870668 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491f44c3-65a7-4f8b-9e6a-7b8d362c4bd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:41.871676 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491f44c3-65a7-4f8b-9e6a-7b8d362c4bd5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:41.871950 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491f44c3-65a7-4f8b-9e6a-7b8d362c4bd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:41.876726 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491f44c3-65a7-4f8b-9e6a-7b8d362c4bd5 admin admin] PATH_INFO: `/v3/projects/b7472f6c889a4f5d90ad3ea58f2321f3` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:41.878346 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491f44c3-65a7-4f8b-9e6a-7b8d362c4bd5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b7472f6c889a4f5d90ad3ea58f2321f3)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:41.880337 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491f44c3-65a7-4f8b-9e6a-7b8d362c4bd5 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:41.928371 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1005/2000] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Jul 25 12:15:41 2022] POST /identity/v3/projects => generated 383 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:15:41.932201 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03ff066-335e-4c4f-920f-f2172941ba61 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:41.948886 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:15:41.987491 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-491f44c3-65a7-4f8b-9e6a-7b8d362c4bd5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b7472f6c889a4f5d90ad3ea58f2321f3', 'request_id': 'req-491f44c3-65a7-4f8b-9e6a-7b8d362c4bd5'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:42.014793 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-491f44c3-65a7-4f8b-9e6a-7b8d362c4bd5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b7472f6c889a4f5d90ad3ea58f2321f3', 'request_id': 'req-491f44c3-65a7-4f8b-9e6a-7b8d362c4bd5'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:42.023259 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-491f44c3-65a7-4f8b-9e6a-7b8d362c4bd5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b7472f6c889a4f5d90ad3ea58f2321f3', 'request_id': 'req-491f44c3-65a7-4f8b-9e6a-7b8d362c4bd5'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:42.043911 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-491f44c3-65a7-4f8b-9e6a-7b8d362c4bd5 admin admin] The token cache is being invalidate because project b7472f6c889a4f5d90ad3ea58f2321f3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 12:15:42.044478 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-491f44c3-65a7-4f8b-9e6a-7b8d362c4bd5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:42.045233 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-491f44c3-65a7-4f8b-9e6a-7b8d362c4bd5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:42.050956 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 996/2001] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Jul 25 12:15:41 2022] DELETE /identity/v3/projects/b7472f6c889a4f5d90ad3ea58f2321f3 => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 12:15:42.052563 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30d7b2f-fb54-47e3-a6fd-ab9ef2a8753d None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:42.053013 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30d7b2f-fb54-47e3-a6fd-ab9ef2a8753d None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:42.053392 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30d7b2f-fb54-47e3-a6fd-ab9ef2a8753d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:42.078300 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-c30d7b2f-fb54-47e3-a6fd-ab9ef2a8753d None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:15:42.082911 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-c30d7b2f-fb54-47e3-a6fd-ab9ef2a8753d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:15:42.122181 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03ff066-335e-4c4f-920f-f2172941ba61 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:42.150640 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:DELETE=4,keystone:SELECT=94,keystone:INSERT=2 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:15:42.185986 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 997/2002] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:15:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:15:42.190429 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc6ea83-abf3-40ee-8eaf-b9ef8a0954a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:42.190926 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e64bd3-97bb-405c-b9ef-3de6f0bf32d4 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:42.190926 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e64bd3-97bb-405c-b9ef-3de6f0bf32d4 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:42.191067 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e64bd3-97bb-405c-b9ef-3de6f0bf32d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:42.191134 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc6ea83-abf3-40ee-8eaf-b9ef8a0954a4 admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:42.191392 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc6ea83-abf3-40ee-8eaf-b9ef8a0954a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:42.191690 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc6ea83-abf3-40ee-8eaf-b9ef8a0954a4 admin admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:42.192387 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc6ea83-abf3-40ee-8eaf-b9ef8a0954a4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:42.194833 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc6ea83-abf3-40ee-8eaf-b9ef8a0954a4 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:42.218964 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-03e64bd3-97bb-405c-b9ef-3de6f0bf32d4 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:15:42.221461 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-03e64bd3-97bb-405c-b9ef-3de6f0bf32d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:15:42.223054 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1006/2003] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Jul 25 12:15:41 2022] POST /identity/v3/users => generated 292 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:15:42.223985 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc6ea83-abf3-40ee-8eaf-b9ef8a0954a4 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:42.226077 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-9dc6ea83-abf3-40ee-8eaf-b9ef8a0954a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:15:42.246101 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc6ea83-abf3-40ee-8eaf-b9ef8a0954a4 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:42.248123 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 998/2004] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:15:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:15:42.249157 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e64bd3-97bb-405c-b9ef-3de6f0bf32d4 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:42.266180 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8bc3a8-e6f1-499f-b6a7-935f6e81875a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:42.266871 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8bc3a8-e6f1-499f-b6a7-935f6e81875a admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:42.267157 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8bc3a8-e6f1-499f-b6a7-935f6e81875a admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:42.268637 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8bc3a8-e6f1-499f-b6a7-935f6e81875a admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:42.269392 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8bc3a8-e6f1-499f-b6a7-935f6e81875a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:42.272210 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8bc3a8-e6f1-499f-b6a7-935f6e81875a admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:42.275870 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e64bd3-97bb-405c-b9ef-3de6f0bf32d4 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:42.287459 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0847afc3-cbbf-464a-9649-9613fe40f856 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:42.288176 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0847afc3-cbbf-464a-9649-9613fe40f856 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:42.288462 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0847afc3-cbbf-464a-9649-9613fe40f856 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:42.288734 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0847afc3-cbbf-464a-9649-9613fe40f856 admin admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:42.289474 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0847afc3-cbbf-464a-9649-9613fe40f856 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:15:42.289782 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0847afc3-cbbf-464a-9649-9613fe40f856 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:42.291753 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0847afc3-cbbf-464a-9649-9613fe40f856 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:42.294585 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1007/2005] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Jul 25 12:15:42 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:15:42.298625 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8bc3a8-e6f1-499f-b6a7-935f6e81875a admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:42.306428 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 999/2006] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Jul 25 12:15:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:15:42.309135 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8bc3a8-e6f1-499f-b6a7-935f6e81875a admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:42.311360 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0847afc3-cbbf-464a-9649-9613fe40f856 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:42.314053 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c59385-09c9-41db-8716-5f365412ff01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:42.314720 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c59385-09c9-41db-8716-5f365412ff01 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:42.315009 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c59385-09c9-41db-8716-5f365412ff01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:42.315297 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c59385-09c9-41db-8716-5f365412ff01 admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:42.316132 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c59385-09c9-41db-8716-5f365412ff01 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:42.317771 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c59385-09c9-41db-8716-5f365412ff01 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:42.323561 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0847afc3-cbbf-464a-9649-9613fe40f856 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:42.328614 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13406d57-155f-499e-a14c-37b757ac18a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:42.329551 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13406d57-155f-499e-a14c-37b757ac18a8 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:42.330063 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13406d57-155f-499e-a14c-37b757ac18a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:42.330549 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1008/2007] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Jul 25 12:15:42 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 12:15:42.331012 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13406d57-155f-499e-a14c-37b757ac18a8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:42.331953 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13406d57-155f-499e-a14c-37b757ac18a8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:42.334962 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13406d57-155f-499e-a14c-37b757ac18a8 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:42.335309 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c59385-09c9-41db-8716-5f365412ff01 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:42.350611 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c59385-09c9-41db-8716-5f365412ff01 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:42.357081 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bdea94-c5be-461a-b298-214cceca1e44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:42.358651 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bdea94-c5be-461a-b298-214cceca1e44 admin admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:42.359300 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bdea94-c5be-461a-b298-214cceca1e44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:42.359889 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bdea94-c5be-461a-b298-214cceca1e44 admin admin] PATH_INFO: `/v3/projects/f63781e2ea3b43b48fc5a065fea7bad4/users/5ea6250801fc4f74b2180533687e8cf2/roles/472313ae69174578824df0bed6390744` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:42.378317 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1000/2008] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Jul 25 12:15:42 2022] POST /identity/v3/projects => generated 371 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:15:42.379722 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13406d57-155f-499e-a14c-37b757ac18a8 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:42.380141 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04bdea94-c5be-461a-b298-214cceca1e44 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f63781e2ea3b43b48fc5a065fea7bad4, user_id=5ea6250801fc4f74b2180533687e8cf2, role_id=472313ae69174578824df0bed6390744)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:42.384979 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04bdea94-c5be-461a-b298-214cceca1e44 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:42.414116 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1009/2009] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Jul 25 12:15:42 2022] PUT /identity/v3/projects/f63781e2ea3b43b48fc5a065fea7bad4/users/5ea6250801fc4f74b2180533687e8cf2/roles/472313ae69174578824df0bed6390744 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 12:15:42.416226 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81757e94-b365-4b1e-a5ae-b5cffefdb317 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:42.417556 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81757e94-b365-4b1e-a5ae-b5cffefdb317 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:42.417852 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81757e94-b365-4b1e-a5ae-b5cffefdb317 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:42.440967 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-81757e94-b365-4b1e-a5ae-b5cffefdb317 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:15:42.442311 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-81757e94-b365-4b1e-a5ae-b5cffefdb317 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:15:42.476646 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13406d57-155f-499e-a14c-37b757ac18a8 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:42.507670 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:DELETE=6,keystone:SELECT=92,keystone:INSERT=2 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:15:42.520715 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1010/2010] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:15:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:15:42.528746 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81757e94-b365-4b1e-a5ae-b5cffefdb317 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:42.531479 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-81757e94-b365-4b1e-a5ae-b5cffefdb317 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:15:42.586295 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6dd303-bd24-463d-a050-fd964720af90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:42.587128 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6dd303-bd24-463d-a050-fd964720af90 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:42.587728 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6dd303-bd24-463d-a050-fd964720af90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:42.588024 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6dd303-bd24-463d-a050-fd964720af90 admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:42.588861 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6dd303-bd24-463d-a050-fd964720af90 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:42.590937 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6dd303-bd24-463d-a050-fd964720af90 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:42.606430 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81757e94-b365-4b1e-a5ae-b5cffefdb317 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:42.612620 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886dfb62-04c6-4da7-be5e-1201df566a6c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51733781201243e8abf9eca39b207750', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:42.613402 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886dfb62-04c6-4da7-be5e-1201df566a6c service neutron] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:42.613779 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886dfb62-04c6-4da7-be5e-1201df566a6c service neutron] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:42.614129 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886dfb62-04c6-4da7-be5e-1201df566a6c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:42.622235 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1001/2011] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Jul 25 12:15:42 2022] POST /identity/v3/users => generated 286 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:15:42.626025 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886dfb62-04c6-4da7-be5e-1201df566a6c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:42.626161 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6dd303-bd24-463d-a050-fd964720af90 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:42.627738 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886dfb62-04c6-4da7-be5e-1201df566a6c service neutron] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:42.637112 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6dd303-bd24-463d-a050-fd964720af90 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:42.640051 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1011/2012] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:15:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:15:42.642998 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02dab01e-0a17-4f1c-bb5b-6df2fb7bbb92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:42.643724 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dab01e-0a17-4f1c-bb5b-6df2fb7bbb92 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:42.644012 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dab01e-0a17-4f1c-bb5b-6df2fb7bbb92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:42.644295 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dab01e-0a17-4f1c-bb5b-6df2fb7bbb92 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:42.645092 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02dab01e-0a17-4f1c-bb5b-6df2fb7bbb92 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:42.646921 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02dab01e-0a17-4f1c-bb5b-6df2fb7bbb92 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:42.659414 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1002/2013] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Jul 25 12:15:42 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 12:15:42.663895 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886dfb62-04c6-4da7-be5e-1201df566a6c service neutron] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:42.674478 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886dfb62-04c6-4da7-be5e-1201df566a6c service neutron] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:42.680878 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff25ad8-c9d6-4727-b056-1c68256ef73c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:42.681596 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff25ad8-c9d6-4727-b056-1c68256ef73c admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:42.681889 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff25ad8-c9d6-4727-b056-1c68256ef73c admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:42.682173 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff25ad8-c9d6-4727-b056-1c68256ef73c admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:42.683365 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff25ad8-c9d6-4727-b056-1c68256ef73c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:42.684549 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff25ad8-c9d6-4727-b056-1c68256ef73c admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:42.696743 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1012/2014] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Jul 25 12:15:42 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 12:15:42.700623 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02dab01e-0a17-4f1c-bb5b-6df2fb7bbb92 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:42.712852 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02dab01e-0a17-4f1c-bb5b-6df2fb7bbb92 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:42.723080 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5e3206-743a-4976-b046-1403684989fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:42.724762 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5e3206-743a-4976-b046-1403684989fd admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:42.724762 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5e3206-743a-4976-b046-1403684989fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:42.724993 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5e3206-743a-4976-b046-1403684989fd admin admin] PATH_INFO: `/v3/projects/91bd04f2c9d34640ab6317fc8dd3dddb/users/5f696b65c8094119af332e7e837b888e/roles/472313ae69174578824df0bed6390744` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:42.747903 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5e3206-743a-4976-b046-1403684989fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=91bd04f2c9d34640ab6317fc8dd3dddb, user_id=5f696b65c8094119af332e7e837b888e, role_id=472313ae69174578824df0bed6390744)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:42.752562 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5e3206-743a-4976-b046-1403684989fd admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:42.785744 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1003/2015] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Jul 25 12:15:42 2022] PUT /identity/v3/projects/91bd04f2c9d34640ab6317fc8dd3dddb/users/5f696b65c8094119af332e7e837b888e/roles/472313ae69174578824df0bed6390744 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 12:15:42.791031 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111bcc1e-74be-4242-a705-d1cbdc00612a None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:42.791512 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111bcc1e-74be-4242-a705-d1cbdc00612a None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:42.791965 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111bcc1e-74be-4242-a705-d1cbdc00612a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:42.800288 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:15:42.816519 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-111bcc1e-74be-4242-a705-d1cbdc00612a None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:15:42.817938 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-111bcc1e-74be-4242-a705-d1cbdc00612a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:15:42.904242 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1013/2016] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:15:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:15:42.915982 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5e3206-743a-4976-b046-1403684989fd admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:42.917918 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-3c5e3206-743a-4976-b046-1403684989fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:15:42.989361 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5e3206-743a-4976-b046-1403684989fd admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:42.995592 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f57556b-1ae4-49e3-a8d4-c2fb5901f727 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51733781201243e8abf9eca39b207750', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:42.996607 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f57556b-1ae4-49e3-a8d4-c2fb5901f727 service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:42.997079 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f57556b-1ae4-49e3-a8d4-c2fb5901f727 service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:42.997560 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f57556b-1ae4-49e3-a8d4-c2fb5901f727 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:43.013238 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f57556b-1ae4-49e3-a8d4-c2fb5901f727 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:43.015735 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f57556b-1ae4-49e3-a8d4-c2fb5901f727 service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:43.029202 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1004/2017] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:15:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:15:44.530220 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed7396c-ecbf-4768-9d34-7be014f43c02 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:44.530656 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed7396c-ecbf-4768-9d34-7be014f43c02 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:44.530841 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed7396c-ecbf-4768-9d34-7be014f43c02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:44.563696 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-9ed7396c-ecbf-4768-9d34-7be014f43c02 None None] MFA Rules not processed for user `5f696b65c8094119af332e7e837b888e`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:15:44.565611 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-9ed7396c-ecbf-4768-9d34-7be014f43c02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:15:44.666802 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1014/2018] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:15:44 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:15:44.675429 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f57556b-1ae4-49e3-a8d4-c2fb5901f727 service neutron] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:44.677715 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-2f57556b-1ae4-49e3-a8d4-c2fb5901f727 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:15:44.815761 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f57556b-1ae4-49e3-a8d4-c2fb5901f727 service neutron] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:44.821363 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4b0790-685d-426c-a1a9-562e1dd5bdcd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f89ce6f0cf04373a414d927fa0f8a7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:44.822127 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4b0790-685d-426c-a1a9-562e1dd5bdcd service cinder] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:44.822484 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4b0790-685d-426c-a1a9-562e1dd5bdcd service cinder] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:44.822832 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4b0790-685d-426c-a1a9-562e1dd5bdcd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:44.837428 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4b0790-685d-426c-a1a9-562e1dd5bdcd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:44.843707 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4b0790-685d-426c-a1a9-562e1dd5bdcd service cinder] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:44.858383 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1005/2019] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:15:44 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:15:44.880903 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed7396c-ecbf-4768-9d34-7be014f43c02 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:44.882432 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-9ed7396c-ecbf-4768-9d34-7be014f43c02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:15:44.908291 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed7396c-ecbf-4768-9d34-7be014f43c02 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:44.914255 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756473c0-fec1-4d93-86f4-a16084e13672 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:44.916595 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756473c0-fec1-4d93-86f4-a16084e13672 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:44.916889 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756473c0-fec1-4d93-86f4-a16084e13672 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:44.917203 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756473c0-fec1-4d93-86f4-a16084e13672 admin admin] PATH_INFO: `/v3/users/01bf176bab8d447d822a30193741b3b5` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:44.918919 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756473c0-fec1-4d93-86f4-a16084e13672 admin admin] RBAC: Authorizing `identity:delete_user(user_id=01bf176bab8d447d822a30193741b3b5)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:44.921200 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756473c0-fec1-4d93-86f4-a16084e13672 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:45.001937 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-756473c0-fec1-4d93-86f4-a16084e13672 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '01bf176bab8d447d822a30193741b3b5'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:45.002613 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-756473c0-fec1-4d93-86f4-a16084e13672 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '01bf176bab8d447d822a30193741b3b5'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:45.014847 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-756473c0-fec1-4d93-86f4-a16084e13672 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '01bf176bab8d447d822a30193741b3b5'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:45.032043 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-756473c0-fec1-4d93-86f4-a16084e13672 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '01bf176bab8d447d822a30193741b3b5'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:45.042295 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-756473c0-fec1-4d93-86f4-a16084e13672 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '01bf176bab8d447d822a30193741b3b5'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:45.050979 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1015/2020] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Jul 25 12:15:44 2022] DELETE /identity/v3/users/01bf176bab8d447d822a30193741b3b5 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:15:45.055344 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a876e3-a75f-4003-9c98-8b2e9b28e669 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:45.055918 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a876e3-a75f-4003-9c98-8b2e9b28e669 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:45.056277 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a876e3-a75f-4003-9c98-8b2e9b28e669 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:45.082736 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-79a876e3-a75f-4003-9c98-8b2e9b28e669 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:15:45.085205 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-79a876e3-a75f-4003-9c98-8b2e9b28e669 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:15:45.104618 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:15:45.177108 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1006/2021] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:15:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 12:15:45.188772 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756473c0-fec1-4d93-86f4-a16084e13672 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:45.190577 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-756473c0-fec1-4d93-86f4-a16084e13672 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:15:45.200516 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:15:45.276156 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756473c0-fec1-4d93-86f4-a16084e13672 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:45.280677 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33703e6-6a60-4a5e-80a0-5f1b3aee8b51 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51733781201243e8abf9eca39b207750', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:45.281398 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33703e6-6a60-4a5e-80a0-5f1b3aee8b51 service neutron] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:45.281635 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33703e6-6a60-4a5e-80a0-5f1b3aee8b51 service neutron] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:45.281908 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33703e6-6a60-4a5e-80a0-5f1b3aee8b51 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:45.309739 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33703e6-6a60-4a5e-80a0-5f1b3aee8b51 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:45.311357 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33703e6-6a60-4a5e-80a0-5f1b3aee8b51 service neutron] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:45.323262 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1016/2022] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:15:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:15:45.642084 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a876e3-a75f-4003-9c98-8b2e9b28e669 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:45.643642 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-79a876e3-a75f-4003-9c98-8b2e9b28e669 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:15:45.666713 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a876e3-a75f-4003-9c98-8b2e9b28e669 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:45.684599 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7a84c9-74e3-4d25-8ba5-21a933dd5f35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:45.685509 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7a84c9-74e3-4d25-8ba5-21a933dd5f35 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:45.685778 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7a84c9-74e3-4d25-8ba5-21a933dd5f35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:45.686054 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7a84c9-74e3-4d25-8ba5-21a933dd5f35 admin admin] PATH_INFO: `/v3/projects/20443f3a0d0346c9be21304cd10335cd` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:45.687340 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7a84c9-74e3-4d25-8ba5-21a933dd5f35 admin admin] RBAC: Authorizing `identity:delete_project(project_id=20443f3a0d0346c9be21304cd10335cd)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:45.689327 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7a84c9-74e3-4d25-8ba5-21a933dd5f35 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:45.745182 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-bc7a84c9-74e3-4d25-8ba5-21a933dd5f35 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '20443f3a0d0346c9be21304cd10335cd', 'request_id': 'req-bc7a84c9-74e3-4d25-8ba5-21a933dd5f35'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:45.752863 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-bc7a84c9-74e3-4d25-8ba5-21a933dd5f35 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '20443f3a0d0346c9be21304cd10335cd', 'request_id': 'req-bc7a84c9-74e3-4d25-8ba5-21a933dd5f35'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:45.762779 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-bc7a84c9-74e3-4d25-8ba5-21a933dd5f35 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '20443f3a0d0346c9be21304cd10335cd', 'request_id': 'req-bc7a84c9-74e3-4d25-8ba5-21a933dd5f35'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:45.785950 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-bc7a84c9-74e3-4d25-8ba5-21a933dd5f35 admin admin] The token cache is being invalidate because project 20443f3a0d0346c9be21304cd10335cd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 12:15:45.786475 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-bc7a84c9-74e3-4d25-8ba5-21a933dd5f35 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:45.787966 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-bc7a84c9-74e3-4d25-8ba5-21a933dd5f35 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:45.791928 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1007/2023] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Jul 25 12:15:45 2022] DELETE /identity/v3/projects/20443f3a0d0346c9be21304cd10335cd => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:15:45.797066 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33703e6-6a60-4a5e-80a0-5f1b3aee8b51 service neutron] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:45.808550 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33703e6-6a60-4a5e-80a0-5f1b3aee8b51 service neutron] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:45.888998 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e355f3d-c232-4fc4-8221-0538abe830bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:45.890509 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e355f3d-c232-4fc4-8221-0538abe830bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:45.890984 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e355f3d-c232-4fc4-8221-0538abe830bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:45.891446 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e355f3d-c232-4fc4-8221-0538abe830bb admin admin] PATH_INFO: `/v3/users/09fb6a302d0c4407b912d8d1985e63b4` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:45.893030 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e355f3d-c232-4fc4-8221-0538abe830bb admin admin] RBAC: Authorizing `identity:delete_user(user_id=09fb6a302d0c4407b912d8d1985e63b4)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:45.896074 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e355f3d-c232-4fc4-8221-0538abe830bb admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:45.967082 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-2e355f3d-c232-4fc4-8221-0538abe830bb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '09fb6a302d0c4407b912d8d1985e63b4'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:45.967971 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-2e355f3d-c232-4fc4-8221-0538abe830bb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '09fb6a302d0c4407b912d8d1985e63b4'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:45.982556 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-2e355f3d-c232-4fc4-8221-0538abe830bb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '09fb6a302d0c4407b912d8d1985e63b4'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:46.000741 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-2e355f3d-c232-4fc4-8221-0538abe830bb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '09fb6a302d0c4407b912d8d1985e63b4'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:46.009596 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-2e355f3d-c232-4fc4-8221-0538abe830bb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '09fb6a302d0c4407b912d8d1985e63b4'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:46.018798 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1017/2024] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Jul 25 12:15:45 2022] DELETE /identity/v3/users/09fb6a302d0c4407b912d8d1985e63b4 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:15:46.419332 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfad903d-d2a7-47b3-8cfe-f259681641d9 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:46.419791 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfad903d-d2a7-47b3-8cfe-f259681641d9 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:46.420217 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfad903d-d2a7-47b3-8cfe-f259681641d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:46.465993 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-dfad903d-d2a7-47b3-8cfe-f259681641d9 None None] MFA Rules not processed for user `5f696b65c8094119af332e7e837b888e`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:15:46.465993 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-dfad903d-d2a7-47b3-8cfe-f259681641d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:15:46.609901 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1008/2025] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:15:46 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:15:46.665265 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e355f3d-c232-4fc4-8221-0538abe830bb admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:46.667339 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-2e355f3d-c232-4fc4-8221-0538abe830bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:15:46.733723 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:15:46.806737 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e355f3d-c232-4fc4-8221-0538abe830bb admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:46.817110 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c442391-6b05-4eea-af0e-76a03519316e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:46.818038 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c442391-6b05-4eea-af0e-76a03519316e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:46.818889 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c442391-6b05-4eea-af0e-76a03519316e admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:46.819326 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c442391-6b05-4eea-af0e-76a03519316e admin admin] PATH_INFO: `/v3/projects/c3bd8c0338434e6c8b64de1c09f78813` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:46.820762 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c442391-6b05-4eea-af0e-76a03519316e admin admin] RBAC: Authorizing `identity:delete_project(project_id=c3bd8c0338434e6c8b64de1c09f78813)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:46.824957 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c442391-6b05-4eea-af0e-76a03519316e admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:46.935220 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-6c442391-6b05-4eea-af0e-76a03519316e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c3bd8c0338434e6c8b64de1c09f78813', 'request_id': 'req-6c442391-6b05-4eea-af0e-76a03519316e'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:46.950831 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-6c442391-6b05-4eea-af0e-76a03519316e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c3bd8c0338434e6c8b64de1c09f78813', 'request_id': 'req-6c442391-6b05-4eea-af0e-76a03519316e'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:46.958509 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-6c442391-6b05-4eea-af0e-76a03519316e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c3bd8c0338434e6c8b64de1c09f78813', 'request_id': 'req-6c442391-6b05-4eea-af0e-76a03519316e'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:46.974607 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-6c442391-6b05-4eea-af0e-76a03519316e admin admin] The token cache is being invalidate because project c3bd8c0338434e6c8b64de1c09f78813 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80298) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 12:15:46.975190 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-6c442391-6b05-4eea-af0e-76a03519316e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:46.975611 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-6c442391-6b05-4eea-af0e-76a03519316e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:46.978026 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1018/2026] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Jul 25 12:15:46 2022] DELETE /identity/v3/projects/c3bd8c0338434e6c8b64de1c09f78813 => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:15:46.982022 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfad903d-d2a7-47b3-8cfe-f259681641d9 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:46.994592 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfad903d-d2a7-47b3-8cfe-f259681641d9 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:47.047394 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:15:47.064676 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe83fbda-c3c5-4fe3-ab3b-8ed40859a268 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:47.065599 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe83fbda-c3c5-4fe3-ab3b-8ed40859a268 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:47.065889 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe83fbda-c3c5-4fe3-ab3b-8ed40859a268 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:47.066171 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe83fbda-c3c5-4fe3-ab3b-8ed40859a268 admin admin] PATH_INFO: `/v3/users/2f3c1eb09f864673a821efca2edf8c27` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:47.067390 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe83fbda-c3c5-4fe3-ab3b-8ed40859a268 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2f3c1eb09f864673a821efca2edf8c27)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:47.070277 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe83fbda-c3c5-4fe3-ab3b-8ed40859a268 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:47.135783 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-fe83fbda-c3c5-4fe3-ab3b-8ed40859a268 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2f3c1eb09f864673a821efca2edf8c27'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:47.146188 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-fe83fbda-c3c5-4fe3-ab3b-8ed40859a268 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2f3c1eb09f864673a821efca2edf8c27'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:47.162412 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-fe83fbda-c3c5-4fe3-ab3b-8ed40859a268 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2f3c1eb09f864673a821efca2edf8c27'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:47.188583 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-fe83fbda-c3c5-4fe3-ab3b-8ed40859a268 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2f3c1eb09f864673a821efca2edf8c27'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:47.189993 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-fe83fbda-c3c5-4fe3-ab3b-8ed40859a268 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2f3c1eb09f864673a821efca2edf8c27'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:47.199012 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1009/2027] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Jul 25 12:15:46 2022] DELETE /identity/v3/users/2f3c1eb09f864673a821efca2edf8c27 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 12:15:47.819462 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c442391-6b05-4eea-af0e-76a03519316e admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:47.821916 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-6c442391-6b05-4eea-af0e-76a03519316e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:15:47.849192 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c442391-6b05-4eea-af0e-76a03519316e admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:47.954810 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a907cc-9255-4f38-ae7d-4079611ff539 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:47.955829 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a907cc-9255-4f38-ae7d-4079611ff539 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:47.956219 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a907cc-9255-4f38-ae7d-4079611ff539 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:47.956602 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a907cc-9255-4f38-ae7d-4079611ff539 admin admin] PATH_INFO: `/v3/projects/711841df205b426ca2f42a7b7e9b77be` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:47.958549 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a907cc-9255-4f38-ae7d-4079611ff539 admin admin] RBAC: Authorizing `identity:delete_project(project_id=711841df205b426ca2f42a7b7e9b77be)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:47.961316 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a907cc-9255-4f38-ae7d-4079611ff539 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:48.063773 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-81a907cc-9255-4f38-ae7d-4079611ff539 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '711841df205b426ca2f42a7b7e9b77be', 'request_id': 'req-81a907cc-9255-4f38-ae7d-4079611ff539'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:48.080359 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-81a907cc-9255-4f38-ae7d-4079611ff539 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '711841df205b426ca2f42a7b7e9b77be', 'request_id': 'req-81a907cc-9255-4f38-ae7d-4079611ff539'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:48.088680 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:15:48.105993 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-81a907cc-9255-4f38-ae7d-4079611ff539 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '711841df205b426ca2f42a7b7e9b77be', 'request_id': 'req-81a907cc-9255-4f38-ae7d-4079611ff539'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:48.143343 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-81a907cc-9255-4f38-ae7d-4079611ff539 admin admin] The token cache is being invalidate because project 711841df205b426ca2f42a7b7e9b77be was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80298) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 12:15:48.143959 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-81a907cc-9255-4f38-ae7d-4079611ff539 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:48.145979 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-81a907cc-9255-4f38-ae7d-4079611ff539 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:48.150807 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1019/2028] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Jul 25 12:15:47 2022] DELETE /identity/v3/projects/711841df205b426ca2f42a7b7e9b77be => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:15:48.158139 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282c7124-234c-4fd3-8052-75c39221b118 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:48.158651 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282c7124-234c-4fd3-8052-75c39221b118 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:48.159148 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282c7124-234c-4fd3-8052-75c39221b118 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:48.189489 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-282c7124-234c-4fd3-8052-75c39221b118 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:15:48.190874 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-282c7124-234c-4fd3-8052-75c39221b118 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:15:48.275274 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1010/2029] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:15:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:15:48.286144 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cebddc3-93a7-4b9c-aa82-79f76cb6090d None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:48.286683 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cebddc3-93a7-4b9c-aa82-79f76cb6090d None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:48.286999 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cebddc3-93a7-4b9c-aa82-79f76cb6090d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:48.312638 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-1cebddc3-93a7-4b9c-aa82-79f76cb6090d None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:15:48.314052 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-1cebddc3-93a7-4b9c-aa82-79f76cb6090d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:15:48.325625 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1020/2030] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:15:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:15:48.330811 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282c7124-234c-4fd3-8052-75c39221b118 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:48.345165 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282c7124-234c-4fd3-8052-75c39221b118 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:48.350328 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93176c7a-1294-4ff2-a484-c2a6e5bf07e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:48.351438 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93176c7a-1294-4ff2-a484-c2a6e5bf07e5 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:48.352176 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93176c7a-1294-4ff2-a484-c2a6e5bf07e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:48.352725 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93176c7a-1294-4ff2-a484-c2a6e5bf07e5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:48.354370 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93176c7a-1294-4ff2-a484-c2a6e5bf07e5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:15:48.354973 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93176c7a-1294-4ff2-a484-c2a6e5bf07e5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:48.357625 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93176c7a-1294-4ff2-a484-c2a6e5bf07e5 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:48.379727 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1011/2031] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Jul 25 12:15:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:15:48.385777 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cebddc3-93a7-4b9c-aa82-79f76cb6090d None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:48.398884 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cebddc3-93a7-4b9c-aa82-79f76cb6090d None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:48.405851 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee685e7-54e5-424e-af31-d79002aaab58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:48.406704 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee685e7-54e5-424e-af31-d79002aaab58 admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:48.407143 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee685e7-54e5-424e-af31-d79002aaab58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:48.407688 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee685e7-54e5-424e-af31-d79002aaab58 admin admin] PATH_INFO: `/v3/projects` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:48.408374 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee685e7-54e5-424e-af31-d79002aaab58 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:48.410560 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee685e7-54e5-424e-af31-d79002aaab58 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:48.452486 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93176c7a-1294-4ff2-a484-c2a6e5bf07e5 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:48.460512 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1021/2032] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Jul 25 12:15:48 2022] POST /identity/v3/projects => generated 385 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:15:48.471015 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93176c7a-1294-4ff2-a484-c2a6e5bf07e5 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:48.565071 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:15:48.574567 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd389e2c-5769-4b13-a2af-17ab5de1d12e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:48.575885 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd389e2c-5769-4b13-a2af-17ab5de1d12e admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:48.576948 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd389e2c-5769-4b13-a2af-17ab5de1d12e admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:48.577401 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd389e2c-5769-4b13-a2af-17ab5de1d12e admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:48.578133 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd389e2c-5769-4b13-a2af-17ab5de1d12e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:48.581923 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd389e2c-5769-4b13-a2af-17ab5de1d12e admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:48.625347 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1012/2033] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Jul 25 12:15:48 2022] POST /identity/v3/users => generated 293 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:15:48.630092 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee685e7-54e5-424e-af31-d79002aaab58 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:48.651109 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee685e7-54e5-424e-af31-d79002aaab58 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:48.658192 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e914d955-8cd8-47fe-8088-bbe1ce2e24e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:48.658977 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e914d955-8cd8-47fe-8088-bbe1ce2e24e7 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:48.659332 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e914d955-8cd8-47fe-8088-bbe1ce2e24e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:48.659956 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e914d955-8cd8-47fe-8088-bbe1ce2e24e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:48.660772 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e914d955-8cd8-47fe-8088-bbe1ce2e24e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:48.662786 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e914d955-8cd8-47fe-8088-bbe1ce2e24e7 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:48.674831 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1022/2034] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Jul 25 12:15:48 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:15:48.679202 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd389e2c-5769-4b13-a2af-17ab5de1d12e admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:48.693185 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd389e2c-5769-4b13-a2af-17ab5de1d12e admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:48.704312 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbb3a39-0464-475c-af61-e21e2a30ecf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:48.705101 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbb3a39-0464-475c-af61-e21e2a30ecf8 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:48.705384 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbb3a39-0464-475c-af61-e21e2a30ecf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:48.705661 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbb3a39-0464-475c-af61-e21e2a30ecf8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:48.706415 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbb3a39-0464-475c-af61-e21e2a30ecf8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:48.708081 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbb3a39-0464-475c-af61-e21e2a30ecf8 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:48.730934 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1013/2035] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Jul 25 12:15:48 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:15:48.731051 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e914d955-8cd8-47fe-8088-bbe1ce2e24e7 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:48.732546 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd3acb1-7e3a-49de-9f01-18fb792f9bc1 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:48.732890 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd3acb1-7e3a-49de-9f01-18fb792f9bc1 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:48.733246 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd3acb1-7e3a-49de-9f01-18fb792f9bc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:48.739880 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e914d955-8cd8-47fe-8088-bbe1ce2e24e7 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:48.745241 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a69a8f-07a8-4550-b92d-5d4e2c4ba7f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:48.746134 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a69a8f-07a8-4550-b92d-5d4e2c4ba7f0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:48.746420 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a69a8f-07a8-4550-b92d-5d4e2c4ba7f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:48.746828 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a69a8f-07a8-4550-b92d-5d4e2c4ba7f0 admin admin] PATH_INFO: `/v3/projects/16b7aebb0cf24b76bec6653792429375/users/124272d04391437db295264af29c7f20/roles/472313ae69174578824df0bed6390744` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:48.763637 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-7fd3acb1-7e3a-49de-9f01-18fb792f9bc1 None None] MFA Rules not processed for user `5ea6250801fc4f74b2180533687e8cf2`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:15:48.765267 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-7fd3acb1-7e3a-49de-9f01-18fb792f9bc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:15:48.765382 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a69a8f-07a8-4550-b92d-5d4e2c4ba7f0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=16b7aebb0cf24b76bec6653792429375, user_id=124272d04391437db295264af29c7f20, role_id=472313ae69174578824df0bed6390744)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:48.772022 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a69a8f-07a8-4550-b92d-5d4e2c4ba7f0 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:48.795680 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1023/2036] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Jul 25 12:15:48 2022] PUT /identity/v3/projects/16b7aebb0cf24b76bec6653792429375/users/124272d04391437db295264af29c7f20/roles/472313ae69174578824df0bed6390744 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:15:48.797661 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1f37a2-752e-486f-944e-2ee99a883f98 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:48.798447 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1f37a2-752e-486f-944e-2ee99a883f98 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:48.798742 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1f37a2-752e-486f-944e-2ee99a883f98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:48.835421 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-2d1f37a2-752e-486f-944e-2ee99a883f98 None None] MFA Rules not processed for user `124272d04391437db295264af29c7f20`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:15:48.838174 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-2d1f37a2-752e-486f-944e-2ee99a883f98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:15:48.848991 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1014/2037] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:15:48 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:15:48.853581 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd3acb1-7e3a-49de-9f01-18fb792f9bc1 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:48.855013 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-7fd3acb1-7e3a-49de-9f01-18fb792f9bc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:15:48.873436 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd3acb1-7e3a-49de-9f01-18fb792f9bc1 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:48.914541 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1024/2038] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:15:48 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:15:48.926501 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1f37a2-752e-486f-944e-2ee99a883f98 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:48.927984 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-2d1f37a2-752e-486f-944e-2ee99a883f98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:15:48.938024 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00f6ee6-ca12-4ac9-aac2-c2182f3d1454 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:48.938743 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00f6ee6-ca12-4ac9-aac2-c2182f3d1454 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:48.939034 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00f6ee6-ca12-4ac9-aac2-c2182f3d1454 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:48.939319 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00f6ee6-ca12-4ac9-aac2-c2182f3d1454 admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:48.940085 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00f6ee6-ca12-4ac9-aac2-c2182f3d1454 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:48.941995 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00f6ee6-ca12-4ac9-aac2-c2182f3d1454 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:48.982679 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1015/2039] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Jul 25 12:15:48 2022] POST /identity/v3/projects => generated 383 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 12:15:48.983959 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00f6ee6-ca12-4ac9-aac2-c2182f3d1454 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:48.990849 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1f37a2-752e-486f-944e-2ee99a883f98 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:49.018713 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=97,keystone:DELETE=1,keystone:INSERT=2 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:15:49.052637 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386b7f14-fe7a-421d-be84-6a8a24aba9ab service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f89ce6f0cf04373a414d927fa0f8a7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:49.053296 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386b7f14-fe7a-421d-be84-6a8a24aba9ab service cinder] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:49.053591 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386b7f14-fe7a-421d-be84-6a8a24aba9ab service cinder] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:49.053878 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386b7f14-fe7a-421d-be84-6a8a24aba9ab service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:49.059722 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:15:49.086340 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00f6ee6-ca12-4ac9-aac2-c2182f3d1454 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:49.092127 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bc0c47-2417-4d31-a9ef-9bdc338ad2eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:49.092813 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bc0c47-2417-4d31-a9ef-9bdc338ad2eb admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:49.093518 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bc0c47-2417-4d31-a9ef-9bdc338ad2eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:49.093518 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bc0c47-2417-4d31-a9ef-9bdc338ad2eb admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:49.094057 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2bc0c47-2417-4d31-a9ef-9bdc338ad2eb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:49.096445 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2bc0c47-2417-4d31-a9ef-9bdc338ad2eb admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:49.139389 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1016/2040] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Jul 25 12:15:48 2022] POST /identity/v3/users => generated 298 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 12:15:49.143454 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bc0c47-2417-4d31-a9ef-9bdc338ad2eb admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:49.156791 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bc0c47-2417-4d31-a9ef-9bdc338ad2eb admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:49.157789 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386b7f14-fe7a-421d-be84-6a8a24aba9ab service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:49.160160 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386b7f14-fe7a-421d-be84-6a8a24aba9ab service cinder] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:49.162146 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548806d2-1cbb-443c-b731-4950623ef0e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:49.162833 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548806d2-1cbb-443c-b731-4950623ef0e8 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:49.163127 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548806d2-1cbb-443c-b731-4950623ef0e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:49.163407 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548806d2-1cbb-443c-b731-4950623ef0e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:49.164191 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548806d2-1cbb-443c-b731-4950623ef0e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:49.165819 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548806d2-1cbb-443c-b731-4950623ef0e8 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:49.180886 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1017/2041] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Jul 25 12:15:49 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:15:49.181280 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1025/2042] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:15:48 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 254 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:15:49.181967 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548806d2-1cbb-443c-b731-4950623ef0e8 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:49.194084 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548806d2-1cbb-443c-b731-4950623ef0e8 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:49.199049 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9bfb79-d175-457a-8899-a8677e591f14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:49.200009 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9bfb79-d175-457a-8899-a8677e591f14 admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:49.200293 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9bfb79-d175-457a-8899-a8677e591f14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:49.200574 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9bfb79-d175-457a-8899-a8677e591f14 admin admin] PATH_INFO: `/v3/projects/ff7fb61a835d438692cf3f64a1c552e1/users/457790bb4d2c4699801c4d51c660d9fa/roles/a7d20885b6d94928a34b3cc5b3a00006` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:49.223196 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9bfb79-d175-457a-8899-a8677e591f14 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ff7fb61a835d438692cf3f64a1c552e1, user_id=457790bb4d2c4699801c4d51c660d9fa, role_id=a7d20885b6d94928a34b3cc5b3a00006)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:49.226194 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9bfb79-d175-457a-8899-a8677e591f14 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:49.245340 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1018/2043] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Jul 25 12:15:49 2022] PUT /identity/v3/projects/ff7fb61a835d438692cf3f64a1c552e1/users/457790bb4d2c4699801c4d51c660d9fa/roles/a7d20885b6d94928a34b3cc5b3a00006 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:15:49.249019 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386b7f14-fe7a-421d-be84-6a8a24aba9ab service cinder] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:49.261571 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386b7f14-fe7a-421d-be84-6a8a24aba9ab service cinder] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:49.331732 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5279ba0-fda6-452c-93fd-8122fa07ee72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:49.332499 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5279ba0-fda6-452c-93fd-8122fa07ee72 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:49.332702 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5279ba0-fda6-452c-93fd-8122fa07ee72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:49.332985 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5279ba0-fda6-452c-93fd-8122fa07ee72 admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:49.333784 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5279ba0-fda6-452c-93fd-8122fa07ee72 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:49.335431 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5279ba0-fda6-452c-93fd-8122fa07ee72 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:49.354074 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1026/2044] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Jul 25 12:15:49 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:15:49.355175 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9bfb79-d175-457a-8899-a8677e591f14 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:49.366217 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9bfb79-d175-457a-8899-a8677e591f14 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:49.370976 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a72481-3985-4981-b9ed-80b730a8cc81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:49.371891 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a72481-3985-4981-b9ed-80b730a8cc81 admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:49.372173 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a72481-3985-4981-b9ed-80b730a8cc81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:49.372451 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a72481-3985-4981-b9ed-80b730a8cc81 admin admin] PATH_INFO: `/v3/projects/ff7fb61a835d438692cf3f64a1c552e1/users/457790bb4d2c4699801c4d51c660d9fa/roles/a7d20885b6d94928a34b3cc5b3a00006` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:49.374584 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a72481-3985-4981-b9ed-80b730a8cc81 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ff7fb61a835d438692cf3f64a1c552e1, user_id=457790bb4d2c4699801c4d51c660d9fa, role_id=a7d20885b6d94928a34b3cc5b3a00006)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:49.377305 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a72481-3985-4981-b9ed-80b730a8cc81 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:49.388571 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1019/2045] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Jul 25 12:15:49 2022] PUT /identity/v3/projects/ff7fb61a835d438692cf3f64a1c552e1/users/457790bb4d2c4699801c4d51c660d9fa/roles/a7d20885b6d94928a34b3cc5b3a00006 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:15:51.902470 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5279ba0-fda6-452c-93fd-8122fa07ee72 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:51.922501 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5279ba0-fda6-452c-93fd-8122fa07ee72 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:52.036543 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e614942a-75f7-4814-8cf4-91cdca2026a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:52.038874 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e614942a-75f7-4814-8cf4-91cdca2026a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:52.039380 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e614942a-75f7-4814-8cf4-91cdca2026a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:52.039936 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e614942a-75f7-4814-8cf4-91cdca2026a6 admin admin] PATH_INFO: `/v3/users/124272d04391437db295264af29c7f20` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:52.041441 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e614942a-75f7-4814-8cf4-91cdca2026a6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=124272d04391437db295264af29c7f20)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:52.043809 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e614942a-75f7-4814-8cf4-91cdca2026a6 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:52.076704 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:15:52.129439 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-e614942a-75f7-4814-8cf4-91cdca2026a6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '124272d04391437db295264af29c7f20'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:52.131159 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-e614942a-75f7-4814-8cf4-91cdca2026a6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '124272d04391437db295264af29c7f20'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:52.154976 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-e614942a-75f7-4814-8cf4-91cdca2026a6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '124272d04391437db295264af29c7f20'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:52.181177 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-e614942a-75f7-4814-8cf4-91cdca2026a6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '124272d04391437db295264af29c7f20'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:52.200496 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-e614942a-75f7-4814-8cf4-91cdca2026a6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '124272d04391437db295264af29c7f20'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:52.214290 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1027/2046] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Jul 25 12:15:51 2022] DELETE /identity/v3/users/124272d04391437db295264af29c7f20 => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:15:52.224988 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803603ab-207c-4a71-8a17-83498e12994d None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:52.226052 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803603ab-207c-4a71-8a17-83498e12994d None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:52.226659 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803603ab-207c-4a71-8a17-83498e12994d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:52.258188 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-803603ab-207c-4a71-8a17-83498e12994d None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:15:52.259770 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-803603ab-207c-4a71-8a17-83498e12994d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:15:52.341039 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1020/2047] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:15:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:15:52.352913 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e614942a-75f7-4814-8cf4-91cdca2026a6 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:52.354377 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-e614942a-75f7-4814-8cf4-91cdca2026a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:15:52.438119 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e614942a-75f7-4814-8cf4-91cdca2026a6 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:52.443045 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d8ae43-eb99-4aa6-840b-a11bc9b28c91 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51733781201243e8abf9eca39b207750', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:52.443935 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d8ae43-eb99-4aa6-840b-a11bc9b28c91 service neutron] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:52.444234 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d8ae43-eb99-4aa6-840b-a11bc9b28c91 service neutron] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:52.444517 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d8ae43-eb99-4aa6-840b-a11bc9b28c91 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:52.457113 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d8ae43-eb99-4aa6-840b-a11bc9b28c91 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:52.459675 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d8ae43-eb99-4aa6-840b-a11bc9b28c91 service neutron] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:52.506946 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1028/2048] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:15:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 12:15:53.043830 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803603ab-207c-4a71-8a17-83498e12994d None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:53.045882 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-803603ab-207c-4a71-8a17-83498e12994d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:15:53.073520 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803603ab-207c-4a71-8a17-83498e12994d None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:53.093867 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e41531-a398-4e36-b676-a8231daec58a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:53.095459 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e41531-a398-4e36-b676-a8231daec58a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:53.095806 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e41531-a398-4e36-b676-a8231daec58a admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:53.096094 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e41531-a398-4e36-b676-a8231daec58a admin admin] PATH_INFO: `/v3/projects/16b7aebb0cf24b76bec6653792429375` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:53.098363 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e41531-a398-4e36-b676-a8231daec58a admin admin] RBAC: Authorizing `identity:delete_project(project_id=16b7aebb0cf24b76bec6653792429375)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:53.101616 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e41531-a398-4e36-b676-a8231daec58a admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:53.309964 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-74e41531-a398-4e36-b676-a8231daec58a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '16b7aebb0cf24b76bec6653792429375', 'request_id': 'req-74e41531-a398-4e36-b676-a8231daec58a'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:53.340688 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-74e41531-a398-4e36-b676-a8231daec58a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '16b7aebb0cf24b76bec6653792429375', 'request_id': 'req-74e41531-a398-4e36-b676-a8231daec58a'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:53.377732 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-74e41531-a398-4e36-b676-a8231daec58a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '16b7aebb0cf24b76bec6653792429375', 'request_id': 'req-74e41531-a398-4e36-b676-a8231daec58a'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:53.399415 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-74e41531-a398-4e36-b676-a8231daec58a admin admin] The token cache is being invalidate because project 16b7aebb0cf24b76bec6653792429375 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 12:15:53.399774 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-74e41531-a398-4e36-b676-a8231daec58a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:53.400464 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-74e41531-a398-4e36-b676-a8231daec58a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:15:53.406293 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1021/2049] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Jul 25 12:15:53 2022] DELETE /identity/v3/projects/16b7aebb0cf24b76bec6653792429375 => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:15:53.407954 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20124959-0b1f-448d-a74e-943465faa0e2 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:53.408450 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20124959-0b1f-448d-a74e-943465faa0e2 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:53.408742 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20124959-0b1f-448d-a74e-943465faa0e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:53.445140 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-20124959-0b1f-448d-a74e-943465faa0e2 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:15:53.448431 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-20124959-0b1f-448d-a74e-943465faa0e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:15:53.565977 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1029/2050] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:15:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:15:53.573952 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d55ec7-c116-4303-9aaf-4df02cfa5a98 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:53.574250 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d55ec7-c116-4303-9aaf-4df02cfa5a98 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:53.574612 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d55ec7-c116-4303-9aaf-4df02cfa5a98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:53.601020 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-c9d55ec7-c116-4303-9aaf-4df02cfa5a98 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:15:53.602935 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-c9d55ec7-c116-4303-9aaf-4df02cfa5a98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:15:53.620649 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20124959-0b1f-448d-a74e-943465faa0e2 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:53.620874 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1022/2051] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:15:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:15:53.642300 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20124959-0b1f-448d-a74e-943465faa0e2 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:53.651716 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c52df1-1804-4f8e-8899-9cb7d198ef87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:53.653354 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c52df1-1804-4f8e-8899-9cb7d198ef87 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:53.653704 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c52df1-1804-4f8e-8899-9cb7d198ef87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:53.654302 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c52df1-1804-4f8e-8899-9cb7d198ef87 admin admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:53.655402 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c52df1-1804-4f8e-8899-9cb7d198ef87 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:15:53.655859 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c52df1-1804-4f8e-8899-9cb7d198ef87 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:53.658201 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c52df1-1804-4f8e-8899-9cb7d198ef87 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:53.702444 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1030/2052] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Jul 25 12:15:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 12:15:53.706506 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d55ec7-c116-4303-9aaf-4df02cfa5a98 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:53.723628 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d55ec7-c116-4303-9aaf-4df02cfa5a98 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:53.732684 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3403ca53-442e-45ea-afd8-872496661e1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:53.734376 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3403ca53-442e-45ea-afd8-872496661e1e admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:53.734856 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3403ca53-442e-45ea-afd8-872496661e1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:53.735348 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3403ca53-442e-45ea-afd8-872496661e1e admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:53.737079 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3403ca53-442e-45ea-afd8-872496661e1e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:53.741370 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3403ca53-442e-45ea-afd8-872496661e1e admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:53.748923 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=87,keystone:INSERT=6,keystone:DELETE=7 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:15:53.810028 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1023/2053] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Jul 25 12:15:53 2022] POST /identity/v3/projects => generated 391 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:15:53.811476 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c52df1-1804-4f8e-8899-9cb7d198ef87 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:53.837195 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c52df1-1804-4f8e-8899-9cb7d198ef87 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:53.898517 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:15:53.966627 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452377c5-365f-4ebf-bb25-9d845cec76ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:53.967348 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452377c5-365f-4ebf-bb25-9d845cec76ab admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:53.967664 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452377c5-365f-4ebf-bb25-9d845cec76ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:53.967978 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452377c5-365f-4ebf-bb25-9d845cec76ab admin admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:53.968664 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452377c5-365f-4ebf-bb25-9d845cec76ab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:53.971762 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452377c5-365f-4ebf-bb25-9d845cec76ab admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:54.014223 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1031/2054] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Jul 25 12:15:53 2022] POST /identity/v3/users => generated 296 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:15:54.019328 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3403ca53-442e-45ea-afd8-872496661e1e admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:54.044756 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3403ca53-442e-45ea-afd8-872496661e1e admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:54.051222 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc4fc6c-51b7-4fa4-9de2-07db2384420e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:54.052548 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc4fc6c-51b7-4fa4-9de2-07db2384420e admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:54.053037 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc4fc6c-51b7-4fa4-9de2-07db2384420e admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:54.053844 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc4fc6c-51b7-4fa4-9de2-07db2384420e admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:54.054911 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc4fc6c-51b7-4fa4-9de2-07db2384420e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:54.057065 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc4fc6c-51b7-4fa4-9de2-07db2384420e admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:54.070342 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1024/2055] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Jul 25 12:15:54 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:15:54.074773 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452377c5-365f-4ebf-bb25-9d845cec76ab admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:54.089809 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452377c5-365f-4ebf-bb25-9d845cec76ab admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:54.094911 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688f5193-61be-4545-b1e1-12c05252cfa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:54.095647 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688f5193-61be-4545-b1e1-12c05252cfa1 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:54.095941 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688f5193-61be-4545-b1e1-12c05252cfa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:54.096222 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688f5193-61be-4545-b1e1-12c05252cfa1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:54.096961 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688f5193-61be-4545-b1e1-12c05252cfa1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:54.098580 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688f5193-61be-4545-b1e1-12c05252cfa1 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:54.116925 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc4fc6c-51b7-4fa4-9de2-07db2384420e admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:54.124724 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1032/2056] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Jul 25 12:15:54 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:15:54.128326 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc4fc6c-51b7-4fa4-9de2-07db2384420e admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:54.133221 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd2c2c7-cf73-458a-8abd-ba3912915597 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:54.134867 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd2c2c7-cf73-458a-8abd-ba3912915597 admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:54.135161 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd2c2c7-cf73-458a-8abd-ba3912915597 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:54.135460 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd2c2c7-cf73-458a-8abd-ba3912915597 admin admin] PATH_INFO: `/v3/projects/a40849c6b22f4b288542cb6080f81127/users/0b56556c3c0c44b898ccd76c9609d625/roles/472313ae69174578824df0bed6390744` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:54.156819 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd2c2c7-cf73-458a-8abd-ba3912915597 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a40849c6b22f4b288542cb6080f81127, user_id=0b56556c3c0c44b898ccd76c9609d625, role_id=472313ae69174578824df0bed6390744)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:54.162372 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd2c2c7-cf73-458a-8abd-ba3912915597 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:54.191488 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1025/2057] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Jul 25 12:15:54 2022] PUT /identity/v3/projects/a40849c6b22f4b288542cb6080f81127/users/0b56556c3c0c44b898ccd76c9609d625/roles/472313ae69174578824df0bed6390744 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:15:54.205803 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0dd511-915d-43c1-b134-fc91d3f14c41 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:54.209446 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0dd511-915d-43c1-b134-fc91d3f14c41 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:54.209760 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0dd511-915d-43c1-b134-fc91d3f14c41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:54.257727 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-af0dd511-915d-43c1-b134-fc91d3f14c41 None None] MFA Rules not processed for user `0b56556c3c0c44b898ccd76c9609d625`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:15:54.258979 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-af0dd511-915d-43c1-b134-fc91d3f14c41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:15:54.358951 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1033/2058] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:15:54 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:15:54.367889 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd2c2c7-cf73-458a-8abd-ba3912915597 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:54.370828 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-9bd2c2c7-cf73-458a-8abd-ba3912915597 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:15:54.557775 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd2c2c7-cf73-458a-8abd-ba3912915597 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:54.575888 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fc25d8-e527-487f-8ef6-48b855f2a04c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f89ce6f0cf04373a414d927fa0f8a7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:54.578833 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fc25d8-e527-487f-8ef6-48b855f2a04c service cinder] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:54.579223 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fc25d8-e527-487f-8ef6-48b855f2a04c service cinder] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:54.579680 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fc25d8-e527-487f-8ef6-48b855f2a04c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:54.608914 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60fc25d8-e527-487f-8ef6-48b855f2a04c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:54.612922 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60fc25d8-e527-487f-8ef6-48b855f2a04c service cinder] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:54.640422 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1026/2059] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:15:54 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 274 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 12:15:57.202893 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62528c94-25c2-4964-8017-af07127a8d99 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:57.203387 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62528c94-25c2-4964-8017-af07127a8d99 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:57.203479 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62528c94-25c2-4964-8017-af07127a8d99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:57.236828 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-62528c94-25c2-4964-8017-af07127a8d99 None None] MFA Rules not processed for user `457790bb4d2c4699801c4d51c660d9fa`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:15:57.238065 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-62528c94-25c2-4964-8017-af07127a8d99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:15:57.322682 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1034/2060] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:15:57 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:15:57.335029 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fc25d8-e527-487f-8ef6-48b855f2a04c service cinder] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:57.336483 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-60fc25d8-e527-487f-8ef6-48b855f2a04c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:15:57.420159 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fc25d8-e527-487f-8ef6-48b855f2a04c service cinder] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:57.424136 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e432f71-4d1b-4ada-9662-5ed054f52593 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ea16528b1374bae9cf54603f7907bfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:57.424778 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e432f71-4d1b-4ada-9662-5ed054f52593 service glance] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:57.425070 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e432f71-4d1b-4ada-9662-5ed054f52593 service glance] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:57.425353 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e432f71-4d1b-4ada-9662-5ed054f52593 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:57.427582 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-3e432f71-4d1b-4ada-9662-5ed054f52593 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:15:57.477760 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:15:57.495862 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e432f71-4d1b-4ada-9662-5ed054f52593 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:57.497477 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e432f71-4d1b-4ada-9662-5ed054f52593 service glance] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:57.509730 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1027/2061] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:15:57 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:15:57.699119 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee983e3e-2b44-4672-95e8-f77305ed4705 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:57.699418 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee983e3e-2b44-4672-95e8-f77305ed4705 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:57.699752 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee983e3e-2b44-4672-95e8-f77305ed4705 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:57.722333 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-ee983e3e-2b44-4672-95e8-f77305ed4705 None None] MFA Rules not processed for user `93207b858d294d049fa56d45810a40aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:15:57.723570 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-ee983e3e-2b44-4672-95e8-f77305ed4705 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:15:57.745497 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:15:57.803033 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1035/2062] 10.222.0.42 () {58 vars in 1009 bytes} [Mon Jul 25 12:15:57 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:15:57.819380 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e432f71-4d1b-4ada-9662-5ed054f52593 service glance] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:15:57.822071 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-3e432f71-4d1b-4ada-9662-5ed054f52593 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:15:57.928999 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e432f71-4d1b-4ada-9662-5ed054f52593 service glance] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:15:57.939030 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b75bce-0045-4655-8ee4-c446edb7fc91 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bd782ea66364eeb96592096eae63d4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:15:57.939735 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b75bce-0045-4655-8ee4-c446edb7fc91 service swift] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:15:57.940201 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b75bce-0045-4655-8ee4-c446edb7fc91 service swift] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:15:57.940498 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b75bce-0045-4655-8ee4-c446edb7fc91 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:15:57.970365 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b75bce-0045-4655-8ee4-c446edb7fc91 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:15:57.970365 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b75bce-0045-4655-8ee4-c446edb7fc91 service swift] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:15:57.977889 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1028/2063] 10.222.0.42 () {60 vars in 1384 bytes} [Mon Jul 25 12:15:57 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 25 12:16:02.599212 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5224ab72-e502-4f9a-9468-faf032f304d6 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:16:02.599976 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5224ab72-e502-4f9a-9468-faf032f304d6 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:16:02.600211 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5224ab72-e502-4f9a-9468-faf032f304d6 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:16:02.602564 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1036/2064] 10.222.0.42 () {54 vars in 921 bytes} [Mon Jul 25 12:16:02 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:16:02.624637 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14b101b-fef0-40fb-ab34-f738c97165fc None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:16:02.625163 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14b101b-fef0-40fb-ab34-f738c97165fc None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:16:02.625278 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14b101b-fef0-40fb-ab34-f738c97165fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:16:02.671064 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-d14b101b-fef0-40fb-ab34-f738c97165fc None None] MFA Rules not processed for user `8012afab6cac4358ab878732e58d0689`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:16:02.672614 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-d14b101b-fef0-40fb-ab34-f738c97165fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:16:02.736649 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5224ab72-e502-4f9a-9468-faf032f304d6 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:16:02.740026 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-5224ab72-e502-4f9a-9468-faf032f304d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:16:02.767656 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1029/2065] 10.222.0.42 () {58 vars in 1018 bytes} [Mon Jul 25 12:16:02 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:16:02.791867 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14b101b-fef0-40fb-ab34-f738c97165fc None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:16:02.793847 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-d14b101b-fef0-40fb-ab34-f738c97165fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:16:02.810549 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14b101b-fef0-40fb-ab34-f738c97165fc None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:16:02.816222 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba23c0e-3226-4ff0-aaf5-e6f3ec3d5fe9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8012afab6cac4358ab878732e58d0689', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:16:02.816983 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba23c0e-3226-4ff0-aaf5-e6f3ec3d5fe9 service placement] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:16:02.817958 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba23c0e-3226-4ff0-aaf5-e6f3ec3d5fe9 service placement] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:16:02.818315 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba23c0e-3226-4ff0-aaf5-e6f3ec3d5fe9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:16:02.827798 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba23c0e-3226-4ff0-aaf5-e6f3ec3d5fe9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:16:02.829506 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba23c0e-3226-4ff0-aaf5-e6f3ec3d5fe9 service placement] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:16:02.855276 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1030/2066] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:16:02 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 12:16:02.865512 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5224ab72-e502-4f9a-9468-faf032f304d6 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:16:02.875214 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b5ea3a-97ec-4c5e-8232-07d1354c48fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:16:02.877440 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b5ea3a-97ec-4c5e-8232-07d1354c48fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:16:02.877601 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b5ea3a-97ec-4c5e-8232-07d1354c48fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:16:02.878621 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b5ea3a-97ec-4c5e-8232-07d1354c48fb admin admin] PATH_INFO: `/v3/users/89a96a9ce9db422f96927b7f59606998` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:16:02.882087 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b5ea3a-97ec-4c5e-8232-07d1354c48fb admin admin] RBAC: Authorizing `identity:delete_user(user_id=89a96a9ce9db422f96927b7f59606998)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:16:02.886840 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b5ea3a-97ec-4c5e-8232-07d1354c48fb admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:16:02.975042 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-d6b5ea3a-97ec-4c5e-8232-07d1354c48fb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '89a96a9ce9db422f96927b7f59606998'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:16:02.976028 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-d6b5ea3a-97ec-4c5e-8232-07d1354c48fb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '89a96a9ce9db422f96927b7f59606998'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:16:02.996607 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-d6b5ea3a-97ec-4c5e-8232-07d1354c48fb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '89a96a9ce9db422f96927b7f59606998'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:16:03.028469 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-d6b5ea3a-97ec-4c5e-8232-07d1354c48fb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '89a96a9ce9db422f96927b7f59606998'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:16:03.037215 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-d6b5ea3a-97ec-4c5e-8232-07d1354c48fb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '89a96a9ce9db422f96927b7f59606998'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:16:03.045918 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1037/2067] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Jul 25 12:16:02 2022] DELETE /identity/v3/users/89a96a9ce9db422f96927b7f59606998 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:16:03.268305 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba23c0e-3226-4ff0-aaf5-e6f3ec3d5fe9 service placement] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:16:03.288534 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba23c0e-3226-4ff0-aaf5-e6f3ec3d5fe9 service placement] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:16:03.393425 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d129cada-9262-4896-b0fd-2c6802611b76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:16:03.394579 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d129cada-9262-4896-b0fd-2c6802611b76 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:16:03.395023 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d129cada-9262-4896-b0fd-2c6802611b76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:16:03.395658 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d129cada-9262-4896-b0fd-2c6802611b76 admin admin] PATH_INFO: `/v3/projects/05e8cdf9803c4388a79e123894ab34b1` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:16:03.397305 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d129cada-9262-4896-b0fd-2c6802611b76 admin admin] RBAC: Authorizing `identity:delete_project(project_id=05e8cdf9803c4388a79e123894ab34b1)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:16:03.399467 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d129cada-9262-4896-b0fd-2c6802611b76 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:16:03.401345 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:16:03.456139 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-d129cada-9262-4896-b0fd-2c6802611b76 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '05e8cdf9803c4388a79e123894ab34b1', 'request_id': 'req-d129cada-9262-4896-b0fd-2c6802611b76'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:16:03.468041 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-d129cada-9262-4896-b0fd-2c6802611b76 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '05e8cdf9803c4388a79e123894ab34b1', 'request_id': 'req-d129cada-9262-4896-b0fd-2c6802611b76'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:16:03.476140 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-d129cada-9262-4896-b0fd-2c6802611b76 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '05e8cdf9803c4388a79e123894ab34b1', 'request_id': 'req-d129cada-9262-4896-b0fd-2c6802611b76'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:16:03.491766 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-d129cada-9262-4896-b0fd-2c6802611b76 admin admin] The token cache is being invalidate because project 05e8cdf9803c4388a79e123894ab34b1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 12:16:03.492334 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-d129cada-9262-4896-b0fd-2c6802611b76 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:16:03.493085 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-d129cada-9262-4896-b0fd-2c6802611b76 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:16:03.494670 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1031/2068] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Jul 25 12:16:03 2022] DELETE /identity/v3/projects/05e8cdf9803c4388a79e123894ab34b1 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:16:03.499838 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d477271e-7430-492e-8203-3904dcd18376 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:16:03.501328 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d477271e-7430-492e-8203-3904dcd18376 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:16:03.501328 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d477271e-7430-492e-8203-3904dcd18376 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:16:03.523296 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-d477271e-7430-492e-8203-3904dcd18376 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:16:03.524569 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-d477271e-7430-492e-8203-3904dcd18376 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:16:03.577456 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:16:03.604791 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1038/2069] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:16:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:16:03.612638 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c76de15-025b-422c-b4a5-ba6bc08c9dad None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:16:03.613201 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c76de15-025b-422c-b4a5-ba6bc08c9dad None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:16:03.615126 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c76de15-025b-422c-b4a5-ba6bc08c9dad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:16:03.639819 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-4c76de15-025b-422c-b4a5-ba6bc08c9dad None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:16:03.642107 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-4c76de15-025b-422c-b4a5-ba6bc08c9dad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:16:03.651411 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1032/2070] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:16:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 12:16:03.656220 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d477271e-7430-492e-8203-3904dcd18376 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:16:03.667644 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d477271e-7430-492e-8203-3904dcd18376 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:16:03.672732 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8395a5-f711-4a6f-a6ca-356add52ea61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:16:03.673401 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8395a5-f711-4a6f-a6ca-356add52ea61 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:16:03.673688 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8395a5-f711-4a6f-a6ca-356add52ea61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:16:03.676284 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8395a5-f711-4a6f-a6ca-356add52ea61 admin admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:16:03.676284 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8395a5-f711-4a6f-a6ca-356add52ea61 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:16:03.676284 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8395a5-f711-4a6f-a6ca-356add52ea61 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:16:03.676815 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8395a5-f711-4a6f-a6ca-356add52ea61 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:16:03.694525 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1039/2071] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Jul 25 12:16:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:16:03.695468 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c76de15-025b-422c-b4a5-ba6bc08c9dad None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:16:03.705747 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c76de15-025b-422c-b4a5-ba6bc08c9dad None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:16:03.712277 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5d34b7-290d-4d7f-a517-6c50d35b98a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:16:03.712842 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5d34b7-290d-4d7f-a517-6c50d35b98a3 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:16:03.713138 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5d34b7-290d-4d7f-a517-6c50d35b98a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:16:03.713431 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5d34b7-290d-4d7f-a517-6c50d35b98a3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:16:03.714295 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5d34b7-290d-4d7f-a517-6c50d35b98a3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:16:03.716295 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5d34b7-290d-4d7f-a517-6c50d35b98a3 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:16:03.749722 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1033/2072] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Jul 25 12:16:03 2022] POST /identity/v3/projects => generated 367 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:16:03.751529 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8395a5-f711-4a6f-a6ca-356add52ea61 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:16:03.764337 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8395a5-f711-4a6f-a6ca-356add52ea61 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:16:03.840413 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafea271-42b3-4a20-b0d0-f071d6e0101b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:16:03.841162 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafea271-42b3-4a20-b0d0-f071d6e0101b admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:16:03.841511 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafea271-42b3-4a20-b0d0-f071d6e0101b admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:16:03.841861 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafea271-42b3-4a20-b0d0-f071d6e0101b admin admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:16:03.842643 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafea271-42b3-4a20-b0d0-f071d6e0101b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:16:03.844686 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafea271-42b3-4a20-b0d0-f071d6e0101b admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:16:03.869369 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1040/2073] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Jul 25 12:16:03 2022] POST /identity/v3/users => generated 284 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 12:16:03.874224 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5d34b7-290d-4d7f-a517-6c50d35b98a3 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:16:03.889071 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5d34b7-290d-4d7f-a517-6c50d35b98a3 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:16:03.893957 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea3ceda-82b2-4bcc-a01e-f8f55bdeeb69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:16:03.894703 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea3ceda-82b2-4bcc-a01e-f8f55bdeeb69 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:16:03.894990 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea3ceda-82b2-4bcc-a01e-f8f55bdeeb69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:16:03.896837 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea3ceda-82b2-4bcc-a01e-f8f55bdeeb69 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:16:03.899173 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea3ceda-82b2-4bcc-a01e-f8f55bdeeb69 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:16:03.900500 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea3ceda-82b2-4bcc-a01e-f8f55bdeeb69 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:16:03.913575 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1034/2074] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Jul 25 12:16:03 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:16:03.917409 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafea271-42b3-4a20-b0d0-f071d6e0101b admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:16:03.928209 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafea271-42b3-4a20-b0d0-f071d6e0101b admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:16:03.933151 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c1c996-667b-418c-b761-b3c3aa1b8586 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:16:03.933822 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c1c996-667b-418c-b761-b3c3aa1b8586 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:16:03.934112 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c1c996-667b-418c-b761-b3c3aa1b8586 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:16:03.934396 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c1c996-667b-418c-b761-b3c3aa1b8586 admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:16:03.935133 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c1c996-667b-418c-b761-b3c3aa1b8586 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:16:03.936785 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c1c996-667b-418c-b761-b3c3aa1b8586 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:16:03.951579 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1041/2075] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Jul 25 12:16:03 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 12:16:03.953998 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea3ceda-82b2-4bcc-a01e-f8f55bdeeb69 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:16:03.965031 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea3ceda-82b2-4bcc-a01e-f8f55bdeeb69 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:16:03.971632 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69ec82f-526f-4583-a8ea-dbd559d917af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:16:03.972620 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69ec82f-526f-4583-a8ea-dbd559d917af admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:16:03.972979 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69ec82f-526f-4583-a8ea-dbd559d917af admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:16:03.973375 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69ec82f-526f-4583-a8ea-dbd559d917af admin admin] PATH_INFO: `/v3/projects/5a56f65324f14737aec1c2e9251706de/users/78a836dfb10f4c2ba6f239ae5bbdfaf6/roles/472313ae69174578824df0bed6390744` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:16:03.991769 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69ec82f-526f-4583-a8ea-dbd559d917af admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a56f65324f14737aec1c2e9251706de, user_id=78a836dfb10f4c2ba6f239ae5bbdfaf6, role_id=472313ae69174578824df0bed6390744)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:16:03.994548 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69ec82f-526f-4583-a8ea-dbd559d917af admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:16:04.014549 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1035/2076] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Jul 25 12:16:03 2022] PUT /identity/v3/projects/5a56f65324f14737aec1c2e9251706de/users/78a836dfb10f4c2ba6f239ae5bbdfaf6/roles/472313ae69174578824df0bed6390744 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:16:04.025248 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778a8f70-047e-4806-a8ee-62ad77848edb None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:16:04.025553 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778a8f70-047e-4806-a8ee-62ad77848edb None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:16:04.025835 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778a8f70-047e-4806-a8ee-62ad77848edb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:16:04.064790 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-778a8f70-047e-4806-a8ee-62ad77848edb None None] MFA Rules not processed for user `78a836dfb10f4c2ba6f239ae5bbdfaf6`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:16:04.066623 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-778a8f70-047e-4806-a8ee-62ad77848edb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:16:04.152945 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1042/2077] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:16:04 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:16:04.164503 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69ec82f-526f-4583-a8ea-dbd559d917af admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:16:04.166368 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-f69ec82f-526f-4583-a8ea-dbd559d917af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:16:04.269218 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69ec82f-526f-4583-a8ea-dbd559d917af admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:16:04.274570 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f910a6-3b50-4ee6-bae1-cb4ef8938f35 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f89ce6f0cf04373a414d927fa0f8a7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:16:04.275311 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f910a6-3b50-4ee6-bae1-cb4ef8938f35 service cinder] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:16:04.275714 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f910a6-3b50-4ee6-bae1-cb4ef8938f35 service cinder] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:16:04.276084 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f910a6-3b50-4ee6-bae1-cb4ef8938f35 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:16:04.287430 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f910a6-3b50-4ee6-bae1-cb4ef8938f35 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:16:04.289105 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f910a6-3b50-4ee6-bae1-cb4ef8938f35 service cinder] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:16:04.301424 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1036/2078] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:16:04 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:16:09.296994 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc242434-90be-4cfc-83a6-18449cdffd3e None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:16:09.297585 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc242434-90be-4cfc-83a6-18449cdffd3e None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:16:09.297770 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc242434-90be-4cfc-83a6-18449cdffd3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:16:09.318680 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-cc242434-90be-4cfc-83a6-18449cdffd3e None None] MFA Rules not processed for user `5ea6250801fc4f74b2180533687e8cf2`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:16:09.319947 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-cc242434-90be-4cfc-83a6-18449cdffd3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:16:09.363465 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:16:09.389332 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1043/2079] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:16:09 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:16:14.296907 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=77,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:16:14.426526 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9faae89-569c-4707-872c-bfddab5065c3 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:16:14.426895 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9faae89-569c-4707-872c-bfddab5065c3 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:16:14.427328 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9faae89-569c-4707-872c-bfddab5065c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:16:14.447465 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-c9faae89-569c-4707-872c-bfddab5065c3 None None] MFA Rules not processed for user `93207b858d294d049fa56d45810a40aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:16:14.448708 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-c9faae89-569c-4707-872c-bfddab5065c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:16:14.523001 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1037/2080] 10.222.0.42 () {58 vars in 1009 bytes} [Mon Jul 25 12:16:14 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:16:14.538784 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc242434-90be-4cfc-83a6-18449cdffd3e None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:16:14.541127 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-cc242434-90be-4cfc-83a6-18449cdffd3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:16:14.624806 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc242434-90be-4cfc-83a6-18449cdffd3e None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:16:14.629871 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc761e2-f19e-435c-913a-4d068adb80b2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bd782ea66364eeb96592096eae63d4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:16:14.630528 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc761e2-f19e-435c-913a-4d068adb80b2 service swift] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:16:14.630819 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc761e2-f19e-435c-913a-4d068adb80b2 service swift] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:16:14.631126 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc761e2-f19e-435c-913a-4d068adb80b2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:16:14.639294 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc761e2-f19e-435c-913a-4d068adb80b2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:16:14.640945 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc761e2-f19e-435c-913a-4d068adb80b2 service swift] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:16:14.649462 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1044/2081] 10.222.0.42 () {60 vars in 1384 bytes} [Mon Jul 25 12:16:14 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 25 12:16:15.281934 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9faae89-569c-4707-872c-bfddab5065c3 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:16:15.281934 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-c9faae89-569c-4707-872c-bfddab5065c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:16:15.433016 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9faae89-569c-4707-872c-bfddab5065c3 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:16:15.448448 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365281c5-02f9-4860-b5a5-ff04c07738bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:16:15.451584 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365281c5-02f9-4860-b5a5-ff04c07738bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:16:15.451869 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365281c5-02f9-4860-b5a5-ff04c07738bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:16:15.452474 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365281c5-02f9-4860-b5a5-ff04c07738bd admin admin] PATH_INFO: `/v3/users/0b56556c3c0c44b898ccd76c9609d625` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:16:15.455532 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365281c5-02f9-4860-b5a5-ff04c07738bd admin admin] RBAC: Authorizing `identity:delete_user(user_id=0b56556c3c0c44b898ccd76c9609d625)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:16:15.459175 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365281c5-02f9-4860-b5a5-ff04c07738bd admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:16:15.562834 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-365281c5-02f9-4860-b5a5-ff04c07738bd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0b56556c3c0c44b898ccd76c9609d625'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:16:15.589475 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-365281c5-02f9-4860-b5a5-ff04c07738bd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0b56556c3c0c44b898ccd76c9609d625'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:16:15.589475 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-365281c5-02f9-4860-b5a5-ff04c07738bd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0b56556c3c0c44b898ccd76c9609d625'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:16:15.623710 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-365281c5-02f9-4860-b5a5-ff04c07738bd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0b56556c3c0c44b898ccd76c9609d625'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:16:15.629839 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-365281c5-02f9-4860-b5a5-ff04c07738bd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0b56556c3c0c44b898ccd76c9609d625'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:16:15.646266 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1038/2082] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Jul 25 12:16:15 2022] DELETE /identity/v3/users/0b56556c3c0c44b898ccd76c9609d625 => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:16:15.647687 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5c4487-0909-489a-8be2-322bf66b9530 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:16:15.648606 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5c4487-0909-489a-8be2-322bf66b9530 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:16:15.649066 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5c4487-0909-489a-8be2-322bf66b9530 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:16:15.682855 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-0b5c4487-0909-489a-8be2-322bf66b9530 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:16:15.684370 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-0b5c4487-0909-489a-8be2-322bf66b9530 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:16:15.778441 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1045/2083] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:16:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 12:16:15.803322 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365281c5-02f9-4860-b5a5-ff04c07738bd admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:16:15.805249 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-365281c5-02f9-4860-b5a5-ff04c07738bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:16:15.868203 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:16:15.948905 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365281c5-02f9-4860-b5a5-ff04c07738bd admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:16:15.953009 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ef66b3-1519-4c1a-bfb4-cfe97b6d1d80 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51733781201243e8abf9eca39b207750', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:16:15.953911 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ef66b3-1519-4c1a-bfb4-cfe97b6d1d80 service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:16:15.954307 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ef66b3-1519-4c1a-bfb4-cfe97b6d1d80 service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:16:15.954681 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ef66b3-1519-4c1a-bfb4-cfe97b6d1d80 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:16:15.972581 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ef66b3-1519-4c1a-bfb4-cfe97b6d1d80 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:16:15.975127 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ef66b3-1519-4c1a-bfb4-cfe97b6d1d80 service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:16:15.997774 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1039/2084] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:16:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:16:16.476836 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5c4487-0909-489a-8be2-322bf66b9530 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:16:16.478648 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-0b5c4487-0909-489a-8be2-322bf66b9530 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:16:16.516816 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5c4487-0909-489a-8be2-322bf66b9530 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:16:16.537038 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45147a7b-cf1d-4dfd-a14b-6e3feef5580d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:16:16.538813 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45147a7b-cf1d-4dfd-a14b-6e3feef5580d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:16:16.539252 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45147a7b-cf1d-4dfd-a14b-6e3feef5580d admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:16:16.539711 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45147a7b-cf1d-4dfd-a14b-6e3feef5580d admin admin] PATH_INFO: `/v3/projects/a40849c6b22f4b288542cb6080f81127` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:16:16.541246 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45147a7b-cf1d-4dfd-a14b-6e3feef5580d admin admin] RBAC: Authorizing `identity:delete_project(project_id=a40849c6b22f4b288542cb6080f81127)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:16:16.543444 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45147a7b-cf1d-4dfd-a14b-6e3feef5580d admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:16:16.646766 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ef66b3-1519-4c1a-bfb4-cfe97b6d1d80 service neutron] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:16:16.648589 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-b9ef66b3-1519-4c1a-bfb4-cfe97b6d1d80 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:16:16.651062 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-45147a7b-cf1d-4dfd-a14b-6e3feef5580d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a40849c6b22f4b288542cb6080f81127', 'request_id': 'req-45147a7b-cf1d-4dfd-a14b-6e3feef5580d'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:16:16.666724 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-45147a7b-cf1d-4dfd-a14b-6e3feef5580d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a40849c6b22f4b288542cb6080f81127', 'request_id': 'req-45147a7b-cf1d-4dfd-a14b-6e3feef5580d'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:16:16.675492 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-45147a7b-cf1d-4dfd-a14b-6e3feef5580d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a40849c6b22f4b288542cb6080f81127', 'request_id': 'req-45147a7b-cf1d-4dfd-a14b-6e3feef5580d'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:16:16.680515 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ef66b3-1519-4c1a-bfb4-cfe97b6d1d80 service neutron] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:16:16.686745 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:16:16.695952 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-45147a7b-cf1d-4dfd-a14b-6e3feef5580d admin admin] The token cache is being invalidate because project a40849c6b22f4b288542cb6080f81127 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80298) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 12:16:16.696923 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-45147a7b-cf1d-4dfd-a14b-6e3feef5580d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:16:16.698672 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-45147a7b-cf1d-4dfd-a14b-6e3feef5580d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:16:16.712852 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1046/2085] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Jul 25 12:16:16 2022] DELETE /identity/v3/projects/a40849c6b22f4b288542cb6080f81127 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:16:16.836489 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45e7491-370a-4ac0-9862-fce107b5c2f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:16:16.837427 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45e7491-370a-4ac0-9862-fce107b5c2f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:16:16.837724 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45e7491-370a-4ac0-9862-fce107b5c2f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:16:16.838410 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45e7491-370a-4ac0-9862-fce107b5c2f3 admin admin] PATH_INFO: `/v3/users/78a836dfb10f4c2ba6f239ae5bbdfaf6` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:16:16.840107 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45e7491-370a-4ac0-9862-fce107b5c2f3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=78a836dfb10f4c2ba6f239ae5bbdfaf6)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:16:16.842257 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45e7491-370a-4ac0-9862-fce107b5c2f3 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:16:16.950045 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-c45e7491-370a-4ac0-9862-fce107b5c2f3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '78a836dfb10f4c2ba6f239ae5bbdfaf6'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:16:16.971288 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-c45e7491-370a-4ac0-9862-fce107b5c2f3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '78a836dfb10f4c2ba6f239ae5bbdfaf6'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:16:16.990570 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-c45e7491-370a-4ac0-9862-fce107b5c2f3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '78a836dfb10f4c2ba6f239ae5bbdfaf6'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:16:17.013633 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-c45e7491-370a-4ac0-9862-fce107b5c2f3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '78a836dfb10f4c2ba6f239ae5bbdfaf6'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:16:17.014723 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-c45e7491-370a-4ac0-9862-fce107b5c2f3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '78a836dfb10f4c2ba6f239ae5bbdfaf6'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:16:17.036673 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1040/2086] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Jul 25 12:16:16 2022] DELETE /identity/v3/users/78a836dfb10f4c2ba6f239ae5bbdfaf6 => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 12:16:17.042245 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa060978-c699-4bf7-b2ff-363c6b80ad70 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:16:17.042541 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa060978-c699-4bf7-b2ff-363c6b80ad70 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:16:17.042875 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa060978-c699-4bf7-b2ff-363c6b80ad70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:16:17.086293 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-aa060978-c699-4bf7-b2ff-363c6b80ad70 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:16:17.087595 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-aa060978-c699-4bf7-b2ff-363c6b80ad70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:16:17.259304 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1047/2087] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:16:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:16:17.270543 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45e7491-370a-4ac0-9862-fce107b5c2f3 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:16:17.272144 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-c45e7491-370a-4ac0-9862-fce107b5c2f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:16:17.347630 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:16:17.354616 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45e7491-370a-4ac0-9862-fce107b5c2f3 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:16:17.360017 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1bc4b7-54c4-4922-9339-018bd96469c8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51733781201243e8abf9eca39b207750', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:16:17.360837 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1bc4b7-54c4-4922-9339-018bd96469c8 service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:16:17.362117 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1bc4b7-54c4-4922-9339-018bd96469c8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:16:17.362515 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1bc4b7-54c4-4922-9339-018bd96469c8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:16:17.377695 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1bc4b7-54c4-4922-9339-018bd96469c8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:16:17.377695 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1bc4b7-54c4-4922-9339-018bd96469c8 service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:16:17.411082 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1041/2088] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:16:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:16:17.820333 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa060978-c699-4bf7-b2ff-363c6b80ad70 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:16:17.821841 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-aa060978-c699-4bf7-b2ff-363c6b80ad70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:16:17.842618 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa060978-c699-4bf7-b2ff-363c6b80ad70 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:16:17.860433 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2f7982-fa2b-4eac-8ab1-b82dd7b05c46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:16:17.861405 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2f7982-fa2b-4eac-8ab1-b82dd7b05c46 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:16:17.861737 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2f7982-fa2b-4eac-8ab1-b82dd7b05c46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:16:17.862031 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2f7982-fa2b-4eac-8ab1-b82dd7b05c46 admin admin] PATH_INFO: `/v3/projects/5a56f65324f14737aec1c2e9251706de` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:16:17.864021 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2f7982-fa2b-4eac-8ab1-b82dd7b05c46 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5a56f65324f14737aec1c2e9251706de)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:16:17.866567 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2f7982-fa2b-4eac-8ab1-b82dd7b05c46 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:16:17.925283 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-3a2f7982-fa2b-4eac-8ab1-b82dd7b05c46 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5a56f65324f14737aec1c2e9251706de', 'request_id': 'req-3a2f7982-fa2b-4eac-8ab1-b82dd7b05c46'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:16:17.936215 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-3a2f7982-fa2b-4eac-8ab1-b82dd7b05c46 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5a56f65324f14737aec1c2e9251706de', 'request_id': 'req-3a2f7982-fa2b-4eac-8ab1-b82dd7b05c46'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:16:17.943185 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-3a2f7982-fa2b-4eac-8ab1-b82dd7b05c46 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5a56f65324f14737aec1c2e9251706de', 'request_id': 'req-3a2f7982-fa2b-4eac-8ab1-b82dd7b05c46'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:16:17.955590 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-3a2f7982-fa2b-4eac-8ab1-b82dd7b05c46 admin admin] The token cache is being invalidate because project 5a56f65324f14737aec1c2e9251706de was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80298) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 12:16:17.955909 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-3a2f7982-fa2b-4eac-8ab1-b82dd7b05c46 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:16:17.956578 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-3a2f7982-fa2b-4eac-8ab1-b82dd7b05c46 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:16:17.958184 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1048/2089] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Jul 25 12:16:17 2022] DELETE /identity/v3/projects/5a56f65324f14737aec1c2e9251706de => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:16:17.967443 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d54f282-0895-4b04-bd1d-ce95990591c8 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:16:17.968111 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d54f282-0895-4b04-bd1d-ce95990591c8 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:16:17.968111 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d54f282-0895-4b04-bd1d-ce95990591c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:16:17.993861 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-6d54f282-0895-4b04-bd1d-ce95990591c8 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:16:17.994993 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-6d54f282-0895-4b04-bd1d-ce95990591c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:16:18.087489 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1042/2090] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:16:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:16:18.096265 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebc4665-f44a-4f00-80ae-cae77a476f08 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:16:18.096560 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebc4665-f44a-4f00-80ae-cae77a476f08 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:16:18.096847 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebc4665-f44a-4f00-80ae-cae77a476f08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:16:18.119101 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-9ebc4665-f44a-4f00-80ae-cae77a476f08 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:16:18.120725 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-9ebc4665-f44a-4f00-80ae-cae77a476f08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:16:18.130687 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1049/2091] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:16:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:16:18.135069 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d54f282-0895-4b04-bd1d-ce95990591c8 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:16:18.145266 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d54f282-0895-4b04-bd1d-ce95990591c8 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:16:18.149943 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a56b2c-88bb-48e1-8f14-60263ee65104 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:16:18.150606 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a56b2c-88bb-48e1-8f14-60263ee65104 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:16:18.150893 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a56b2c-88bb-48e1-8f14-60263ee65104 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:16:18.151166 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a56b2c-88bb-48e1-8f14-60263ee65104 admin admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:16:18.151932 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a56b2c-88bb-48e1-8f14-60263ee65104 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:16:18.152246 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a56b2c-88bb-48e1-8f14-60263ee65104 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:16:18.153868 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a56b2c-88bb-48e1-8f14-60263ee65104 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:16:18.173256 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1043/2092] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Jul 25 12:16:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:16:18.174453 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebc4665-f44a-4f00-80ae-cae77a476f08 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:16:18.184629 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebc4665-f44a-4f00-80ae-cae77a476f08 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:16:18.193708 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33264fac-055e-41e3-a543-1dd51e2804e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:16:18.194391 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33264fac-055e-41e3-a543-1dd51e2804e4 admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:16:18.194675 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33264fac-055e-41e3-a543-1dd51e2804e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:16:18.194962 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33264fac-055e-41e3-a543-1dd51e2804e4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:16:18.195687 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33264fac-055e-41e3-a543-1dd51e2804e4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:16:18.197589 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33264fac-055e-41e3-a543-1dd51e2804e4 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:16:18.228558 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1050/2093] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Jul 25 12:16:18 2022] POST /identity/v3/projects => generated 365 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 12:16:18.232560 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a56b2c-88bb-48e1-8f14-60263ee65104 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:16:18.244575 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a56b2c-88bb-48e1-8f14-60263ee65104 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:16:18.368944 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27da9c7c-a9c9-4237-a5f7-f7d1d45bcd80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:16:18.369646 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27da9c7c-a9c9-4237-a5f7-f7d1d45bcd80 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:16:18.369936 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27da9c7c-a9c9-4237-a5f7-f7d1d45bcd80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:16:18.370223 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27da9c7c-a9c9-4237-a5f7-f7d1d45bcd80 admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:16:18.370941 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27da9c7c-a9c9-4237-a5f7-f7d1d45bcd80 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:16:18.373204 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27da9c7c-a9c9-4237-a5f7-f7d1d45bcd80 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:16:18.528974 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1044/2094] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Jul 25 12:16:18 2022] POST /identity/v3/users => generated 283 bytes in 297 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:16:18.533362 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33264fac-055e-41e3-a543-1dd51e2804e4 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:16:18.556371 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33264fac-055e-41e3-a543-1dd51e2804e4 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:16:18.561602 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f570602-3df9-40ec-8cb0-0973329b4094 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:16:18.562361 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f570602-3df9-40ec-8cb0-0973329b4094 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:16:18.562725 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f570602-3df9-40ec-8cb0-0973329b4094 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:16:18.563066 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f570602-3df9-40ec-8cb0-0973329b4094 admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:16:18.564036 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f570602-3df9-40ec-8cb0-0973329b4094 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:16:18.565767 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f570602-3df9-40ec-8cb0-0973329b4094 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:16:18.591235 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1051/2095] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Jul 25 12:16:18 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:16:18.595013 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27da9c7c-a9c9-4237-a5f7-f7d1d45bcd80 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:16:18.610475 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27da9c7c-a9c9-4237-a5f7-f7d1d45bcd80 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:16:18.617203 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e07d88-203e-42b3-b884-b8671a5c6b61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:16:18.618200 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e07d88-203e-42b3-b884-b8671a5c6b61 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:16:18.619094 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e07d88-203e-42b3-b884-b8671a5c6b61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:16:18.619782 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e07d88-203e-42b3-b884-b8671a5c6b61 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:16:18.621611 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e07d88-203e-42b3-b884-b8671a5c6b61 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:16:18.624431 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e07d88-203e-42b3-b884-b8671a5c6b61 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:16:18.640818 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1045/2096] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Jul 25 12:16:18 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:16:18.641753 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f570602-3df9-40ec-8cb0-0973329b4094 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:16:18.654142 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f570602-3df9-40ec-8cb0-0973329b4094 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:16:18.661441 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f43d994-b069-43f6-83c4-b8224b91d2d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:16:18.662577 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f43d994-b069-43f6-83c4-b8224b91d2d1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:16:18.662941 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f43d994-b069-43f6-83c4-b8224b91d2d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:16:18.663309 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f43d994-b069-43f6-83c4-b8224b91d2d1 admin admin] PATH_INFO: `/v3/projects/1309e3631a1740e18b0883766f202f4d/users/212647e2d58c459f8c5085f3f9fdff20/roles/472313ae69174578824df0bed6390744` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:16:18.681357 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f43d994-b069-43f6-83c4-b8224b91d2d1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1309e3631a1740e18b0883766f202f4d, user_id=212647e2d58c459f8c5085f3f9fdff20, role_id=472313ae69174578824df0bed6390744)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:16:18.684184 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f43d994-b069-43f6-83c4-b8224b91d2d1 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:16:18.699177 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1052/2097] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Jul 25 12:16:18 2022] PUT /identity/v3/projects/1309e3631a1740e18b0883766f202f4d/users/212647e2d58c459f8c5085f3f9fdff20/roles/472313ae69174578824df0bed6390744 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:16:18.709899 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a20f1f6-b8aa-41c2-88ae-14b69236e2b0 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:16:18.710201 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a20f1f6-b8aa-41c2-88ae-14b69236e2b0 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:16:18.710481 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a20f1f6-b8aa-41c2-88ae-14b69236e2b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:16:18.745192 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-5a20f1f6-b8aa-41c2-88ae-14b69236e2b0 None None] MFA Rules not processed for user `212647e2d58c459f8c5085f3f9fdff20`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:16:18.746466 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-5a20f1f6-b8aa-41c2-88ae-14b69236e2b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:16:18.800064 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:16:18.848770 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1046/2098] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:16:18 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:16:18.848917 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f43d994-b069-43f6-83c4-b8224b91d2d1 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:16:18.866727 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:16:18.917974 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f43d994-b069-43f6-83c4-b8224b91d2d1 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:16:18.922855 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101d8e6f-52a7-4143-804b-d1b790f719ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:16:18.923530 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101d8e6f-52a7-4143-804b-d1b790f719ba admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:16:18.924463 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101d8e6f-52a7-4143-804b-d1b790f719ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:16:18.924759 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101d8e6f-52a7-4143-804b-d1b790f719ba admin admin] PATH_INFO: `/v3/projects` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:16:18.925482 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101d8e6f-52a7-4143-804b-d1b790f719ba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:16:18.927451 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101d8e6f-52a7-4143-804b-d1b790f719ba admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:16:18.961056 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1053/2099] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Jul 25 12:16:18 2022] POST /identity/v3/projects => generated 365 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 12:16:18.962499 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a20f1f6-b8aa-41c2-88ae-14b69236e2b0 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:16:18.974711 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a20f1f6-b8aa-41c2-88ae-14b69236e2b0 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:16:19.041224 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a001479-6751-4fd6-a121-3bcd1c1924fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:16:19.041826 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a001479-6751-4fd6-a121-3bcd1c1924fd admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:16:19.042128 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a001479-6751-4fd6-a121-3bcd1c1924fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:16:19.042410 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a001479-6751-4fd6-a121-3bcd1c1924fd admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:16:19.043146 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a001479-6751-4fd6-a121-3bcd1c1924fd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:16:19.045139 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a001479-6751-4fd6-a121-3bcd1c1924fd admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:16:19.070484 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1047/2100] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Jul 25 12:16:18 2022] POST /identity/v3/users => generated 289 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:16:19.074652 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101d8e6f-52a7-4143-804b-d1b790f719ba admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:16:19.086941 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101d8e6f-52a7-4143-804b-d1b790f719ba admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:16:19.092327 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbc68c6-b0d9-48d7-a85b-e0ddc8fb044c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:16:19.093005 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbc68c6-b0d9-48d7-a85b-e0ddc8fb044c admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:16:19.093506 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbc68c6-b0d9-48d7-a85b-e0ddc8fb044c admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:16:19.093801 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbc68c6-b0d9-48d7-a85b-e0ddc8fb044c admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:16:19.094605 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbc68c6-b0d9-48d7-a85b-e0ddc8fb044c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:16:19.096356 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbc68c6-b0d9-48d7-a85b-e0ddc8fb044c admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:16:19.108054 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1054/2101] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Jul 25 12:16:19 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:16:19.112406 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a001479-6751-4fd6-a121-3bcd1c1924fd admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:16:19.123033 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a001479-6751-4fd6-a121-3bcd1c1924fd admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:16:19.128080 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc11dbc-eb04-48e8-a369-ac3c7cd7aed5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:16:19.129487 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc11dbc-eb04-48e8-a369-ac3c7cd7aed5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:16:19.129772 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc11dbc-eb04-48e8-a369-ac3c7cd7aed5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:16:19.130049 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc11dbc-eb04-48e8-a369-ac3c7cd7aed5 admin admin] PATH_INFO: `/v3/projects/c85123f4e9334ba9b0ae806e5e34b1e1/users/53e70803a29e45de89ba3501e54c569a/roles/a7d20885b6d94928a34b3cc5b3a00006` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:16:19.148621 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc11dbc-eb04-48e8-a369-ac3c7cd7aed5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c85123f4e9334ba9b0ae806e5e34b1e1, user_id=53e70803a29e45de89ba3501e54c569a, role_id=a7d20885b6d94928a34b3cc5b3a00006)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:16:19.151312 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc11dbc-eb04-48e8-a369-ac3c7cd7aed5 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:16:19.168104 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1048/2102] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Jul 25 12:16:19 2022] PUT /identity/v3/projects/c85123f4e9334ba9b0ae806e5e34b1e1/users/53e70803a29e45de89ba3501e54c569a/roles/a7d20885b6d94928a34b3cc5b3a00006 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 12:16:19.171137 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbc68c6-b0d9-48d7-a85b-e0ddc8fb044c admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:16:19.182969 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbc68c6-b0d9-48d7-a85b-e0ddc8fb044c admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:16:19.249082 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e7d112-9e62-41cd-830e-a374f3f2bfb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:16:19.249850 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e7d112-9e62-41cd-830e-a374f3f2bfb3 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:16:19.250208 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e7d112-9e62-41cd-830e-a374f3f2bfb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:16:19.250558 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e7d112-9e62-41cd-830e-a374f3f2bfb3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:16:19.251352 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e7d112-9e62-41cd-830e-a374f3f2bfb3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:16:19.253045 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e7d112-9e62-41cd-830e-a374f3f2bfb3 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:16:19.270570 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc11dbc-eb04-48e8-a369-ac3c7cd7aed5 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:16:19.276055 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1055/2103] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Jul 25 12:16:19 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 12:16:19.282354 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc11dbc-eb04-48e8-a369-ac3c7cd7aed5 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:16:19.287565 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c87c44-5dda-4d81-a6eb-ba354f93379c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:16:19.288485 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c87c44-5dda-4d81-a6eb-ba354f93379c admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:16:19.288899 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c87c44-5dda-4d81-a6eb-ba354f93379c admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:16:19.289189 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c87c44-5dda-4d81-a6eb-ba354f93379c admin admin] PATH_INFO: `/v3/projects/c85123f4e9334ba9b0ae806e5e34b1e1/users/53e70803a29e45de89ba3501e54c569a/roles/a7d20885b6d94928a34b3cc5b3a00006` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:16:19.291178 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c87c44-5dda-4d81-a6eb-ba354f93379c admin admin] RBAC: Authorizing `identity:create_grant(project_id=c85123f4e9334ba9b0ae806e5e34b1e1, user_id=53e70803a29e45de89ba3501e54c569a, role_id=a7d20885b6d94928a34b3cc5b3a00006)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:16:19.293917 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c87c44-5dda-4d81-a6eb-ba354f93379c admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:16:19.313590 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1049/2104] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Jul 25 12:16:19 2022] PUT /identity/v3/projects/c85123f4e9334ba9b0ae806e5e34b1e1/users/53e70803a29e45de89ba3501e54c569a/roles/a7d20885b6d94928a34b3cc5b3a00006 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:16:19.318346 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bd379d-f8cb-4b4a-8997-e40d400a17e5 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:16:19.318640 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bd379d-f8cb-4b4a-8997-e40d400a17e5 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:16:19.318922 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bd379d-f8cb-4b4a-8997-e40d400a17e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:16:19.349268 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-e8bd379d-f8cb-4b4a-8997-e40d400a17e5 None None] MFA Rules not processed for user `53e70803a29e45de89ba3501e54c569a`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:16:19.350529 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-e8bd379d-f8cb-4b4a-8997-e40d400a17e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:16:19.433595 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1056/2105] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:16:19 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:16:19.445185 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c87c44-5dda-4d81-a6eb-ba354f93379c admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:16:19.447070 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-80c87c44-5dda-4d81-a6eb-ba354f93379c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:16:19.513106 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c87c44-5dda-4d81-a6eb-ba354f93379c admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:16:19.517106 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c08a50-33ab-4b6f-a500-9c0b80c8e49a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f89ce6f0cf04373a414d927fa0f8a7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:16:19.517871 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c08a50-33ab-4b6f-a500-9c0b80c8e49a service cinder] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:16:19.518225 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c08a50-33ab-4b6f-a500-9c0b80c8e49a service cinder] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:16:19.518859 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c08a50-33ab-4b6f-a500-9c0b80c8e49a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:16:19.529533 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c08a50-33ab-4b6f-a500-9c0b80c8e49a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:16:19.530992 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c08a50-33ab-4b6f-a500-9c0b80c8e49a service cinder] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:16:19.545768 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1050/2106] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:16:19 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:16:19.654529 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bd379d-f8cb-4b4a-8997-e40d400a17e5 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:16:19.671734 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bd379d-f8cb-4b4a-8997-e40d400a17e5 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:16:19.676528 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07067c0c-89ed-4030-ba27-84bd16d96841 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f89ce6f0cf04373a414d927fa0f8a7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:16:19.677651 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07067c0c-89ed-4030-ba27-84bd16d96841 service cinder] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:16:19.678227 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07067c0c-89ed-4030-ba27-84bd16d96841 service cinder] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:16:19.678787 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07067c0c-89ed-4030-ba27-84bd16d96841 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:16:19.720371 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:16:19.811217 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07067c0c-89ed-4030-ba27-84bd16d96841 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:16:19.815667 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07067c0c-89ed-4030-ba27-84bd16d96841 service cinder] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:16:19.838111 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1057/2107] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:16:19 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:16:22.729695 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c08a50-33ab-4b6f-a500-9c0b80c8e49a service cinder] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:16:22.771686 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:16:22.793635 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c08a50-33ab-4b6f-a500-9c0b80c8e49a service cinder] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:16:22.798178 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0777a827-0924-46c5-aab3-9ec015827c94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:16:22.799306 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0777a827-0924-46c5-aab3-9ec015827c94 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:16:22.799630 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0777a827-0924-46c5-aab3-9ec015827c94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:16:22.799915 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0777a827-0924-46c5-aab3-9ec015827c94 admin admin] PATH_INFO: `/v3/users/212647e2d58c459f8c5085f3f9fdff20` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:16:22.801119 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0777a827-0924-46c5-aab3-9ec015827c94 admin admin] RBAC: Authorizing `identity:delete_user(user_id=212647e2d58c459f8c5085f3f9fdff20)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:16:22.803050 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0777a827-0924-46c5-aab3-9ec015827c94 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:16:22.850842 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-0777a827-0924-46c5-aab3-9ec015827c94 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '212647e2d58c459f8c5085f3f9fdff20'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:16:22.857725 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-0777a827-0924-46c5-aab3-9ec015827c94 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '212647e2d58c459f8c5085f3f9fdff20'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:16:22.864782 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-0777a827-0924-46c5-aab3-9ec015827c94 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '212647e2d58c459f8c5085f3f9fdff20'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:16:22.877132 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-0777a827-0924-46c5-aab3-9ec015827c94 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '212647e2d58c459f8c5085f3f9fdff20'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:16:22.877793 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-0777a827-0924-46c5-aab3-9ec015827c94 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '212647e2d58c459f8c5085f3f9fdff20'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:16:22.884111 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1051/2108] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Jul 25 12:16:22 2022] DELETE /identity/v3/users/212647e2d58c459f8c5085f3f9fdff20 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:16:22.888544 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e988d3-f904-4a7b-a80c-88bc300775e3 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:16:22.889041 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e988d3-f904-4a7b-a80c-88bc300775e3 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:16:22.889160 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e988d3-f904-4a7b-a80c-88bc300775e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:16:22.908392 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-b0e988d3-f904-4a7b-a80c-88bc300775e3 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:16:22.909884 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-b0e988d3-f904-4a7b-a80c-88bc300775e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:16:22.978682 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1058/2109] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:16:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:16:22.990758 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0777a827-0924-46c5-aab3-9ec015827c94 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:16:22.993946 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-0777a827-0924-46c5-aab3-9ec015827c94 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:16:23.055481 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0777a827-0924-46c5-aab3-9ec015827c94 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:16:23.059086 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e303ad-831f-47bb-bdfc-109bbda89ad0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51733781201243e8abf9eca39b207750', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:16:23.059768 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e303ad-831f-47bb-bdfc-109bbda89ad0 service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:16:23.060059 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e303ad-831f-47bb-bdfc-109bbda89ad0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:16:23.060344 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e303ad-831f-47bb-bdfc-109bbda89ad0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:16:23.069821 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e303ad-831f-47bb-bdfc-109bbda89ad0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:16:23.071262 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e303ad-831f-47bb-bdfc-109bbda89ad0 service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:16:23.080737 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1052/2110] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:16:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:16:23.467530 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e988d3-f904-4a7b-a80c-88bc300775e3 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:16:23.468985 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-b0e988d3-f904-4a7b-a80c-88bc300775e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:16:23.491243 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e988d3-f904-4a7b-a80c-88bc300775e3 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:16:23.497046 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887d07f5-0783-4972-9cb1-2ad160c1c907 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:16:23.497587 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887d07f5-0783-4972-9cb1-2ad160c1c907 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:16:23.497984 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887d07f5-0783-4972-9cb1-2ad160c1c907 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:16:23.498618 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887d07f5-0783-4972-9cb1-2ad160c1c907 admin admin] PATH_INFO: `/v3/projects/1309e3631a1740e18b0883766f202f4d` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:16:23.499737 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887d07f5-0783-4972-9cb1-2ad160c1c907 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1309e3631a1740e18b0883766f202f4d)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:16:23.501780 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887d07f5-0783-4972-9cb1-2ad160c1c907 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:16:23.580924 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-887d07f5-0783-4972-9cb1-2ad160c1c907 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1309e3631a1740e18b0883766f202f4d', 'request_id': 'req-887d07f5-0783-4972-9cb1-2ad160c1c907'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:16:23.596106 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-887d07f5-0783-4972-9cb1-2ad160c1c907 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1309e3631a1740e18b0883766f202f4d', 'request_id': 'req-887d07f5-0783-4972-9cb1-2ad160c1c907'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:16:23.607501 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-887d07f5-0783-4972-9cb1-2ad160c1c907 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1309e3631a1740e18b0883766f202f4d', 'request_id': 'req-887d07f5-0783-4972-9cb1-2ad160c1c907'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:16:23.620123 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-887d07f5-0783-4972-9cb1-2ad160c1c907 admin admin] The token cache is being invalidate because project 1309e3631a1740e18b0883766f202f4d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80298) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 12:16:23.620510 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-887d07f5-0783-4972-9cb1-2ad160c1c907 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:16:23.621217 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-887d07f5-0783-4972-9cb1-2ad160c1c907 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:16:23.623056 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1059/2111] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Jul 25 12:16:23 2022] DELETE /identity/v3/projects/1309e3631a1740e18b0883766f202f4d => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:16:23.626821 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e303ad-831f-47bb-bdfc-109bbda89ad0 service neutron] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:16:23.636788 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e303ad-831f-47bb-bdfc-109bbda89ad0 service neutron] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:16:23.693219 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9074771e-bfca-447f-b110-5c1c262b8ba5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:16:23.694091 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9074771e-bfca-447f-b110-5c1c262b8ba5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:16:23.694379 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9074771e-bfca-447f-b110-5c1c262b8ba5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:16:23.694665 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9074771e-bfca-447f-b110-5c1c262b8ba5 admin admin] PATH_INFO: `/v3/users/53e70803a29e45de89ba3501e54c569a` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:16:23.695902 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9074771e-bfca-447f-b110-5c1c262b8ba5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=53e70803a29e45de89ba3501e54c569a)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:16:23.697841 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9074771e-bfca-447f-b110-5c1c262b8ba5 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:16:23.700111 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:16:23.745240 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-9074771e-bfca-447f-b110-5c1c262b8ba5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '53e70803a29e45de89ba3501e54c569a'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:16:23.752106 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-9074771e-bfca-447f-b110-5c1c262b8ba5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '53e70803a29e45de89ba3501e54c569a'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:16:23.759378 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-9074771e-bfca-447f-b110-5c1c262b8ba5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '53e70803a29e45de89ba3501e54c569a'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:16:23.771508 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-9074771e-bfca-447f-b110-5c1c262b8ba5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '53e70803a29e45de89ba3501e54c569a'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:16:23.772211 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-9074771e-bfca-447f-b110-5c1c262b8ba5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '53e70803a29e45de89ba3501e54c569a'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:16:23.778256 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1053/2112] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Jul 25 12:16:23 2022] DELETE /identity/v3/users/53e70803a29e45de89ba3501e54c569a => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:16:24.055137 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887d07f5-0783-4972-9cb1-2ad160c1c907 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:16:24.057694 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-887d07f5-0783-4972-9cb1-2ad160c1c907 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:16:24.071566 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887d07f5-0783-4972-9cb1-2ad160c1c907 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:16:24.107228 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:16:24.132383 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc02d2c-712e-470d-9c5a-3c98591f529f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:16:24.133241 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc02d2c-712e-470d-9c5a-3c98591f529f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:16:24.133527 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc02d2c-712e-470d-9c5a-3c98591f529f admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:16:24.133804 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc02d2c-712e-470d-9c5a-3c98591f529f admin admin] PATH_INFO: `/v3/projects/c85123f4e9334ba9b0ae806e5e34b1e1` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:16:24.134987 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc02d2c-712e-470d-9c5a-3c98591f529f admin admin] RBAC: Authorizing `identity:delete_project(project_id=c85123f4e9334ba9b0ae806e5e34b1e1)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:16:24.136950 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc02d2c-712e-470d-9c5a-3c98591f529f admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:16:24.182782 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-5cc02d2c-712e-470d-9c5a-3c98591f529f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c85123f4e9334ba9b0ae806e5e34b1e1', 'request_id': 'req-5cc02d2c-712e-470d-9c5a-3c98591f529f'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:16:24.189986 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-5cc02d2c-712e-470d-9c5a-3c98591f529f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c85123f4e9334ba9b0ae806e5e34b1e1', 'request_id': 'req-5cc02d2c-712e-470d-9c5a-3c98591f529f'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:16:24.195975 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-5cc02d2c-712e-470d-9c5a-3c98591f529f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c85123f4e9334ba9b0ae806e5e34b1e1', 'request_id': 'req-5cc02d2c-712e-470d-9c5a-3c98591f529f'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:16:24.208822 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-5cc02d2c-712e-470d-9c5a-3c98591f529f admin admin] The token cache is being invalidate because project c85123f4e9334ba9b0ae806e5e34b1e1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80298) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 12:16:24.209114 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-5cc02d2c-712e-470d-9c5a-3c98591f529f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:16:24.209841 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-5cc02d2c-712e-470d-9c5a-3c98591f529f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:16:24.211509 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1060/2113] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Jul 25 12:16:24 2022] DELETE /identity/v3/projects/c85123f4e9334ba9b0ae806e5e34b1e1 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:16:24.216692 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d77196c-1491-4a59-bd5e-eb21216b5fbd None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:16:24.217234 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d77196c-1491-4a59-bd5e-eb21216b5fbd None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:16:24.217589 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d77196c-1491-4a59-bd5e-eb21216b5fbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:16:24.237125 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-8d77196c-1491-4a59-bd5e-eb21216b5fbd None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:16:24.238333 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-8d77196c-1491-4a59-bd5e-eb21216b5fbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:16:24.296245 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1054/2114] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:16:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:16:24.303996 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e266d29-381b-4f32-b589-f6054b9126ec None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:16:24.304293 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e266d29-381b-4f32-b589-f6054b9126ec None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:16:24.304580 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e266d29-381b-4f32-b589-f6054b9126ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:16:24.324750 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-6e266d29-381b-4f32-b589-f6054b9126ec None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:16:24.325961 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-6e266d29-381b-4f32-b589-f6054b9126ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:16:24.334606 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1061/2115] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:16:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:16:24.338777 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d77196c-1491-4a59-bd5e-eb21216b5fbd None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:16:24.348366 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d77196c-1491-4a59-bd5e-eb21216b5fbd None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:16:24.352976 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05931c2a-cac0-454d-b716-ff1ae2a3dcca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:16:24.353637 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05931c2a-cac0-454d-b716-ff1ae2a3dcca admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:16:24.353928 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05931c2a-cac0-454d-b716-ff1ae2a3dcca admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:16:24.354228 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05931c2a-cac0-454d-b716-ff1ae2a3dcca admin admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:16:24.354946 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05931c2a-cac0-454d-b716-ff1ae2a3dcca admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:16:24.355264 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05931c2a-cac0-454d-b716-ff1ae2a3dcca admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:16:24.356906 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05931c2a-cac0-454d-b716-ff1ae2a3dcca admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:16:24.369229 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1055/2116] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Jul 25 12:16:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:16:24.373559 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e266d29-381b-4f32-b589-f6054b9126ec None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:16:24.383407 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e266d29-381b-4f32-b589-f6054b9126ec None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:16:24.388275 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2398f358-c373-49f9-8752-8c9e65e0a094 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:16:24.388965 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2398f358-c373-49f9-8752-8c9e65e0a094 admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:16:24.389257 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2398f358-c373-49f9-8752-8c9e65e0a094 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:16:24.389562 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2398f358-c373-49f9-8752-8c9e65e0a094 admin admin] PATH_INFO: `/v3/projects` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:16:24.390248 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2398f358-c373-49f9-8752-8c9e65e0a094 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:16:24.392198 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2398f358-c373-49f9-8752-8c9e65e0a094 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:16:24.427102 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1062/2117] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Jul 25 12:16:24 2022] POST /identity/v3/projects => generated 387 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 12:16:24.428286 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05931c2a-cac0-454d-b716-ff1ae2a3dcca admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:16:24.438826 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05931c2a-cac0-454d-b716-ff1ae2a3dcca admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:16:24.505033 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015f6d64-6d3a-4f0c-8bb2-2f7cc17495c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:16:24.505867 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015f6d64-6d3a-4f0c-8bb2-2f7cc17495c3 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:16:24.506159 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015f6d64-6d3a-4f0c-8bb2-2f7cc17495c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:16:24.506443 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015f6d64-6d3a-4f0c-8bb2-2f7cc17495c3 admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:16:24.507188 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015f6d64-6d3a-4f0c-8bb2-2f7cc17495c3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:16:24.509378 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015f6d64-6d3a-4f0c-8bb2-2f7cc17495c3 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:16:24.528537 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:16:24.536171 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1056/2118] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Jul 25 12:16:24 2022] POST /identity/v3/users => generated 294 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 12:16:24.540295 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2398f358-c373-49f9-8752-8c9e65e0a094 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:16:24.552373 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2398f358-c373-49f9-8752-8c9e65e0a094 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:16:24.557079 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e80e3d-ed88-48bd-af3a-b2d9974f7c61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:16:24.557751 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e80e3d-ed88-48bd-af3a-b2d9974f7c61 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:16:24.558044 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e80e3d-ed88-48bd-af3a-b2d9974f7c61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:16:24.558364 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e80e3d-ed88-48bd-af3a-b2d9974f7c61 admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:16:24.559101 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e80e3d-ed88-48bd-af3a-b2d9974f7c61 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:16:24.560733 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e80e3d-ed88-48bd-af3a-b2d9974f7c61 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:16:24.571614 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1063/2119] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Jul 25 12:16:24 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 12:16:24.575434 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015f6d64-6d3a-4f0c-8bb2-2f7cc17495c3 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:16:24.585520 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015f6d64-6d3a-4f0c-8bb2-2f7cc17495c3 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:16:24.590161 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23db20b-fc16-4747-b839-7d44b4fb7355 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:16:24.590817 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23db20b-fc16-4747-b839-7d44b4fb7355 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:16:24.591110 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23db20b-fc16-4747-b839-7d44b4fb7355 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:16:24.591397 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23db20b-fc16-4747-b839-7d44b4fb7355 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:16:24.592206 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23db20b-fc16-4747-b839-7d44b4fb7355 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:16:24.593806 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23db20b-fc16-4747-b839-7d44b4fb7355 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:16:24.610278 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1057/2120] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Jul 25 12:16:24 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 12:16:24.614658 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e80e3d-ed88-48bd-af3a-b2d9974f7c61 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:16:24.624737 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e80e3d-ed88-48bd-af3a-b2d9974f7c61 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:16:24.630018 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52747ebf-85f4-4123-a0bb-9900471e1207 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:16:24.630910 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52747ebf-85f4-4123-a0bb-9900471e1207 admin admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:16:24.631202 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52747ebf-85f4-4123-a0bb-9900471e1207 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:16:24.631487 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52747ebf-85f4-4123-a0bb-9900471e1207 admin admin] PATH_INFO: `/v3/projects/7d32a30efc694b34b88404cc5cacc9a5/users/7c172c877ed2448bae829963c88c6d53/roles/472313ae69174578824df0bed6390744` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:16:24.646571 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52747ebf-85f4-4123-a0bb-9900471e1207 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d32a30efc694b34b88404cc5cacc9a5, user_id=7c172c877ed2448bae829963c88c6d53, role_id=472313ae69174578824df0bed6390744)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:16:24.649287 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52747ebf-85f4-4123-a0bb-9900471e1207 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:16:24.662935 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1064/2121] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Jul 25 12:16:24 2022] PUT /identity/v3/projects/7d32a30efc694b34b88404cc5cacc9a5/users/7c172c877ed2448bae829963c88c6d53/roles/472313ae69174578824df0bed6390744 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:16:24.673769 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a444c0b-32c8-4e16-9099-6c0a37354428 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:16:24.674212 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a444c0b-32c8-4e16-9099-6c0a37354428 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:16:24.674597 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a444c0b-32c8-4e16-9099-6c0a37354428 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:16:24.705925 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-5a444c0b-32c8-4e16-9099-6c0a37354428 None None] MFA Rules not processed for user `7c172c877ed2448bae829963c88c6d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:16:24.707441 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-5a444c0b-32c8-4e16-9099-6c0a37354428 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:16:24.776575 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1058/2122] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:16:24 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:16:24.780958 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52747ebf-85f4-4123-a0bb-9900471e1207 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:16:24.842405 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52747ebf-85f4-4123-a0bb-9900471e1207 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:16:24.847005 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4ba1bc-47d9-4a4f-813f-36bc3649844b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:16:24.847710 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4ba1bc-47d9-4a4f-813f-36bc3649844b admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:16:24.847995 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4ba1bc-47d9-4a4f-813f-36bc3649844b admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:16:24.848272 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4ba1bc-47d9-4a4f-813f-36bc3649844b admin admin] PATH_INFO: `/v3/projects` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:16:24.848952 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4ba1bc-47d9-4a4f-813f-36bc3649844b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:16:24.850848 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4ba1bc-47d9-4a4f-813f-36bc3649844b admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:16:24.859919 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:16:24.877616 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1065/2123] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Jul 25 12:16:24 2022] POST /identity/v3/projects => generated 387 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:16:24.882067 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a444c0b-32c8-4e16-9099-6c0a37354428 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:16:24.892494 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a444c0b-32c8-4e16-9099-6c0a37354428 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:16:24.947489 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff56f583-e726-4f96-832a-dbc5eaa088b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:16:24.948158 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff56f583-e726-4f96-832a-dbc5eaa088b0 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:16:24.948450 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff56f583-e726-4f96-832a-dbc5eaa088b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:16:24.948727 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff56f583-e726-4f96-832a-dbc5eaa088b0 admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:16:24.949406 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff56f583-e726-4f96-832a-dbc5eaa088b0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:16:24.951312 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff56f583-e726-4f96-832a-dbc5eaa088b0 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:16:24.971033 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1059/2124] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Jul 25 12:16:24 2022] POST /identity/v3/users => generated 300 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:16:24.975519 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4ba1bc-47d9-4a4f-813f-36bc3649844b admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:16:24.985764 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4ba1bc-47d9-4a4f-813f-36bc3649844b admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:16:24.991087 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b9f526-307c-4bee-8696-6c59b008cdc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:16:24.991780 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b9f526-307c-4bee-8696-6c59b008cdc9 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:16:24.992084 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b9f526-307c-4bee-8696-6c59b008cdc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:16:24.992354 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b9f526-307c-4bee-8696-6c59b008cdc9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:16:24.993085 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b9f526-307c-4bee-8696-6c59b008cdc9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:16:24.994681 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b9f526-307c-4bee-8696-6c59b008cdc9 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:16:25.005008 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1066/2125] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Jul 25 12:16:24 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:16:25.008910 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff56f583-e726-4f96-832a-dbc5eaa088b0 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:16:25.018679 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff56f583-e726-4f96-832a-dbc5eaa088b0 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:16:25.023733 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2c2399-7946-4fc6-b1a6-661aedadd9fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:16:25.024650 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2c2399-7946-4fc6-b1a6-661aedadd9fd admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:16:25.024969 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2c2399-7946-4fc6-b1a6-661aedadd9fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:16:25.025249 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2c2399-7946-4fc6-b1a6-661aedadd9fd admin admin] PATH_INFO: `/v3/projects/7f8e28bbe5fb499eac78bb0654445cfa/users/b3084d4adcfa40a296d9da28fb3b0ab4/roles/a7d20885b6d94928a34b3cc5b3a00006` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:16:25.040600 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2c2399-7946-4fc6-b1a6-661aedadd9fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f8e28bbe5fb499eac78bb0654445cfa, user_id=b3084d4adcfa40a296d9da28fb3b0ab4, role_id=a7d20885b6d94928a34b3cc5b3a00006)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:16:25.043746 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2c2399-7946-4fc6-b1a6-661aedadd9fd admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:16:25.060968 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1060/2126] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Jul 25 12:16:25 2022] PUT /identity/v3/projects/7f8e28bbe5fb499eac78bb0654445cfa/users/b3084d4adcfa40a296d9da28fb3b0ab4/roles/a7d20885b6d94928a34b3cc5b3a00006 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:16:25.065152 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b9f526-307c-4bee-8696-6c59b008cdc9 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:16:25.078598 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b9f526-307c-4bee-8696-6c59b008cdc9 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:16:25.173030 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6126aa8-bb50-4679-9409-e43369ef9e84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:16:25.174782 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6126aa8-bb50-4679-9409-e43369ef9e84 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:16:25.175309 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6126aa8-bb50-4679-9409-e43369ef9e84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:16:25.176685 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6126aa8-bb50-4679-9409-e43369ef9e84 admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:16:25.177673 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6126aa8-bb50-4679-9409-e43369ef9e84 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:16:25.179536 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6126aa8-bb50-4679-9409-e43369ef9e84 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:16:25.192476 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1067/2127] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Jul 25 12:16:25 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:16:25.196642 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2c2399-7946-4fc6-b1a6-661aedadd9fd admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:16:25.207050 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2c2399-7946-4fc6-b1a6-661aedadd9fd admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:16:25.213041 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc45c6e1-ebeb-46ee-aee9-8329e165d8b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:16:25.214121 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc45c6e1-ebeb-46ee-aee9-8329e165d8b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:16:25.214448 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc45c6e1-ebeb-46ee-aee9-8329e165d8b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:16:25.214863 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc45c6e1-ebeb-46ee-aee9-8329e165d8b3 admin admin] PATH_INFO: `/v3/projects/7f8e28bbe5fb499eac78bb0654445cfa/users/b3084d4adcfa40a296d9da28fb3b0ab4/roles/a7d20885b6d94928a34b3cc5b3a00006` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:16:25.217050 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc45c6e1-ebeb-46ee-aee9-8329e165d8b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f8e28bbe5fb499eac78bb0654445cfa, user_id=b3084d4adcfa40a296d9da28fb3b0ab4, role_id=a7d20885b6d94928a34b3cc5b3a00006)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:16:25.219767 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc45c6e1-ebeb-46ee-aee9-8329e165d8b3 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:16:25.232188 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1061/2128] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Jul 25 12:16:25 2022] PUT /identity/v3/projects/7f8e28bbe5fb499eac78bb0654445cfa/users/b3084d4adcfa40a296d9da28fb3b0ab4/roles/a7d20885b6d94928a34b3cc5b3a00006 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:16:25.245110 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876ce350-bfde-445b-be36-5d05ca67562e None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:16:25.245405 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876ce350-bfde-445b-be36-5d05ca67562e None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:16:25.245689 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876ce350-bfde-445b-be36-5d05ca67562e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:16:25.275521 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-876ce350-bfde-445b-be36-5d05ca67562e None None] MFA Rules not processed for user `b3084d4adcfa40a296d9da28fb3b0ab4`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:16:25.276801 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-876ce350-bfde-445b-be36-5d05ca67562e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:16:25.352552 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1068/2129] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:16:25 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:16:25.366701 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc45c6e1-ebeb-46ee-aee9-8329e165d8b3 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:16:25.368554 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-bc45c6e1-ebeb-46ee-aee9-8329e165d8b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:16:25.404618 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:16:25.437244 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc45c6e1-ebeb-46ee-aee9-8329e165d8b3 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:16:25.440952 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8a6230-ed15-4fa1-b7fd-bb03fa12ad14 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f89ce6f0cf04373a414d927fa0f8a7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:16:25.441608 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8a6230-ed15-4fa1-b7fd-bb03fa12ad14 service cinder] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:16:25.442000 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8a6230-ed15-4fa1-b7fd-bb03fa12ad14 service cinder] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:16:25.442287 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8a6230-ed15-4fa1-b7fd-bb03fa12ad14 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:16:25.451696 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8a6230-ed15-4fa1-b7fd-bb03fa12ad14 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:16:25.453876 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8a6230-ed15-4fa1-b7fd-bb03fa12ad14 service cinder] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:16:25.466294 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1062/2130] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:16:25 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:16:25.497116 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876ce350-bfde-445b-be36-5d05ca67562e None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:16:25.563715 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876ce350-bfde-445b-be36-5d05ca67562e None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:16:25.568431 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75aba5b1-5468-4f26-a45d-dc4644ac4448 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:16:25.569302 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75aba5b1-5468-4f26-a45d-dc4644ac4448 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:16:25.569586 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75aba5b1-5468-4f26-a45d-dc4644ac4448 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:16:25.569867 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75aba5b1-5468-4f26-a45d-dc4644ac4448 admin admin] PATH_INFO: `/v3/users/7c172c877ed2448bae829963c88c6d53` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:16:25.571083 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75aba5b1-5468-4f26-a45d-dc4644ac4448 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7c172c877ed2448bae829963c88c6d53)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:16:25.573013 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75aba5b1-5468-4f26-a45d-dc4644ac4448 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:16:25.575790 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:16:25.628334 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3963c8-7c59-48be-8726-aa3e206ec988 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:16:25.628790 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3963c8-7c59-48be-8726-aa3e206ec988 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:16:25.629172 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3963c8-7c59-48be-8726-aa3e206ec988 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:16:25.646570 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-75aba5b1-5468-4f26-a45d-dc4644ac4448 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7c172c877ed2448bae829963c88c6d53'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:16:25.647354 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-75aba5b1-5468-4f26-a45d-dc4644ac4448 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7c172c877ed2448bae829963c88c6d53'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:16:25.650487 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-ce3963c8-7c59-48be-8726-aa3e206ec988 None None] MFA Rules not processed for user `5f696b65c8094119af332e7e837b888e`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:16:25.652075 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-ce3963c8-7c59-48be-8726-aa3e206ec988 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:16:25.662299 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-75aba5b1-5468-4f26-a45d-dc4644ac4448 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7c172c877ed2448bae829963c88c6d53'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:16:25.689761 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-75aba5b1-5468-4f26-a45d-dc4644ac4448 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7c172c877ed2448bae829963c88c6d53'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:16:25.708135 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-75aba5b1-5468-4f26-a45d-dc4644ac4448 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7c172c877ed2448bae829963c88c6d53'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:16:25.715509 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1069/2131] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Jul 25 12:16:25 2022] DELETE /identity/v3/users/7c172c877ed2448bae829963c88c6d53 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:16:25.720089 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31aa5f00-b716-43c2-bf08-30358598ed5d None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:16:25.720622 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31aa5f00-b716-43c2-bf08-30358598ed5d None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:16:25.720925 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31aa5f00-b716-43c2-bf08-30358598ed5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:16:25.735583 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1063/2132] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:16:25 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 12:16:25.747128 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-31aa5f00-b716-43c2-bf08-30358598ed5d None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:16:25.748532 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-31aa5f00-b716-43c2-bf08-30358598ed5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:16:25.823099 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1070/2133] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:16:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:16:25.832914 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3963c8-7c59-48be-8726-aa3e206ec988 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:16:25.834426 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-ce3963c8-7c59-48be-8726-aa3e206ec988 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:16:25.894940 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3963c8-7c59-48be-8726-aa3e206ec988 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:16:25.951889 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71979638-eeb6-4918-98de-bc2f910c04cb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51733781201243e8abf9eca39b207750', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:16:25.952542 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71979638-eeb6-4918-98de-bc2f910c04cb service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:16:25.952930 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71979638-eeb6-4918-98de-bc2f910c04cb service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:16:25.953212 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71979638-eeb6-4918-98de-bc2f910c04cb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:16:25.963181 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71979638-eeb6-4918-98de-bc2f910c04cb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:16:25.964648 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71979638-eeb6-4918-98de-bc2f910c04cb service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:16:25.974951 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1064/2134] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:16:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:16:26.261047 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31aa5f00-b716-43c2-bf08-30358598ed5d None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:16:26.262443 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-31aa5f00-b716-43c2-bf08-30358598ed5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:16:26.276141 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31aa5f00-b716-43c2-bf08-30358598ed5d None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:16:26.281309 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5245f32f-a6ec-47d3-af37-ac286a19683f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:16:26.282492 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5245f32f-a6ec-47d3-af37-ac286a19683f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:16:26.282809 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5245f32f-a6ec-47d3-af37-ac286a19683f admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:16:26.283090 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5245f32f-a6ec-47d3-af37-ac286a19683f admin admin] PATH_INFO: `/v3/projects/7d32a30efc694b34b88404cc5cacc9a5` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:16:26.285280 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5245f32f-a6ec-47d3-af37-ac286a19683f admin admin] RBAC: Authorizing `identity:delete_project(project_id=7d32a30efc694b34b88404cc5cacc9a5)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:16:26.287262 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5245f32f-a6ec-47d3-af37-ac286a19683f admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:16:26.343727 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-5245f32f-a6ec-47d3-af37-ac286a19683f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7d32a30efc694b34b88404cc5cacc9a5', 'request_id': 'req-5245f32f-a6ec-47d3-af37-ac286a19683f'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:16:26.353511 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-5245f32f-a6ec-47d3-af37-ac286a19683f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7d32a30efc694b34b88404cc5cacc9a5', 'request_id': 'req-5245f32f-a6ec-47d3-af37-ac286a19683f'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:16:26.360876 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-5245f32f-a6ec-47d3-af37-ac286a19683f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7d32a30efc694b34b88404cc5cacc9a5', 'request_id': 'req-5245f32f-a6ec-47d3-af37-ac286a19683f'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:16:26.375005 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-5245f32f-a6ec-47d3-af37-ac286a19683f admin admin] The token cache is being invalidate because project 7d32a30efc694b34b88404cc5cacc9a5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80298) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 12:16:26.375395 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-5245f32f-a6ec-47d3-af37-ac286a19683f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:16:26.376383 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-5245f32f-a6ec-47d3-af37-ac286a19683f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:16:26.378593 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1071/2135] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Jul 25 12:16:26 2022] DELETE /identity/v3/projects/7d32a30efc694b34b88404cc5cacc9a5 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:16:26.383661 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71979638-eeb6-4918-98de-bc2f910c04cb service neutron] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:16:26.393857 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71979638-eeb6-4918-98de-bc2f910c04cb service neutron] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:16:26.429451 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:16:26.459869 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1727e483-a389-481e-ada8-1c89d11c9aef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:16:26.460774 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1727e483-a389-481e-ada8-1c89d11c9aef admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:16:26.461079 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1727e483-a389-481e-ada8-1c89d11c9aef admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:16:26.461343 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1727e483-a389-481e-ada8-1c89d11c9aef admin admin] PATH_INFO: `/v3/users/b3084d4adcfa40a296d9da28fb3b0ab4` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:16:26.462574 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1727e483-a389-481e-ada8-1c89d11c9aef admin admin] RBAC: Authorizing `identity:delete_user(user_id=b3084d4adcfa40a296d9da28fb3b0ab4)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:16:26.464576 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1727e483-a389-481e-ada8-1c89d11c9aef admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:16:26.517911 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-1727e483-a389-481e-ada8-1c89d11c9aef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b3084d4adcfa40a296d9da28fb3b0ab4'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:16:26.525023 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-1727e483-a389-481e-ada8-1c89d11c9aef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b3084d4adcfa40a296d9da28fb3b0ab4'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:16:26.532793 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-1727e483-a389-481e-ada8-1c89d11c9aef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b3084d4adcfa40a296d9da28fb3b0ab4'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:16:26.550280 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-1727e483-a389-481e-ada8-1c89d11c9aef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b3084d4adcfa40a296d9da28fb3b0ab4'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:16:26.551491 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-1727e483-a389-481e-ada8-1c89d11c9aef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b3084d4adcfa40a296d9da28fb3b0ab4'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:16:26.558063 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1065/2136] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Jul 25 12:16:26 2022] DELETE /identity/v3/users/b3084d4adcfa40a296d9da28fb3b0ab4 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:16:26.878088 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5245f32f-a6ec-47d3-af37-ac286a19683f admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:16:26.879577 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-5245f32f-a6ec-47d3-af37-ac286a19683f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:16:26.887858 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:16:26.894433 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5245f32f-a6ec-47d3-af37-ac286a19683f admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:16:26.960623 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0c8beb-f9e6-4d85-bdde-406de368cc9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:16:26.961431 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0c8beb-f9e6-4d85-bdde-406de368cc9c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:16:26.961755 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0c8beb-f9e6-4d85-bdde-406de368cc9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:16:26.962072 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0c8beb-f9e6-4d85-bdde-406de368cc9c admin admin] PATH_INFO: `/v3/projects/7f8e28bbe5fb499eac78bb0654445cfa` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:16:26.963304 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0c8beb-f9e6-4d85-bdde-406de368cc9c admin admin] RBAC: Authorizing `identity:delete_project(project_id=7f8e28bbe5fb499eac78bb0654445cfa)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:16:26.965537 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0c8beb-f9e6-4d85-bdde-406de368cc9c admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:16:27.014518 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-8f0c8beb-f9e6-4d85-bdde-406de368cc9c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7f8e28bbe5fb499eac78bb0654445cfa', 'request_id': 'req-8f0c8beb-f9e6-4d85-bdde-406de368cc9c'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:16:27.022392 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-8f0c8beb-f9e6-4d85-bdde-406de368cc9c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7f8e28bbe5fb499eac78bb0654445cfa', 'request_id': 'req-8f0c8beb-f9e6-4d85-bdde-406de368cc9c'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:16:27.028366 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-8f0c8beb-f9e6-4d85-bdde-406de368cc9c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7f8e28bbe5fb499eac78bb0654445cfa', 'request_id': 'req-8f0c8beb-f9e6-4d85-bdde-406de368cc9c'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:16:27.041495 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-8f0c8beb-f9e6-4d85-bdde-406de368cc9c admin admin] The token cache is being invalidate because project 7f8e28bbe5fb499eac78bb0654445cfa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80298) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 12:16:27.041789 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-8f0c8beb-f9e6-4d85-bdde-406de368cc9c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:16:27.042443 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-8f0c8beb-f9e6-4d85-bdde-406de368cc9c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:16:27.044033 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1072/2137] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Jul 25 12:16:26 2022] DELETE /identity/v3/projects/7f8e28bbe5fb499eac78bb0654445cfa => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:16:27.049022 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54cace4-b05e-42c2-8c86-6367a3801e22 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:16:27.049487 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54cace4-b05e-42c2-8c86-6367a3801e22 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:16:27.049603 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54cace4-b05e-42c2-8c86-6367a3801e22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:16:27.069306 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-e54cace4-b05e-42c2-8c86-6367a3801e22 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:16:27.070517 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-e54cace4-b05e-42c2-8c86-6367a3801e22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:16:27.131897 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1066/2138] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:16:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:16:27.140468 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5255b8ed-8aa4-4c76-8616-275c24a6a2d5 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:16:27.140782 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5255b8ed-8aa4-4c76-8616-275c24a6a2d5 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:16:27.141067 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5255b8ed-8aa4-4c76-8616-275c24a6a2d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:16:27.162866 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-5255b8ed-8aa4-4c76-8616-275c24a6a2d5 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:16:27.164100 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-5255b8ed-8aa4-4c76-8616-275c24a6a2d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:16:27.172510 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1073/2139] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:16:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 12:16:27.177006 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54cace4-b05e-42c2-8c86-6367a3801e22 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:16:27.186897 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54cace4-b05e-42c2-8c86-6367a3801e22 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:16:27.192125 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ca69d1-88b7-4692-ad5f-0d74167bd0d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:16:27.192799 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ca69d1-88b7-4692-ad5f-0d74167bd0d4 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:16:27.193091 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ca69d1-88b7-4692-ad5f-0d74167bd0d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:16:27.193402 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ca69d1-88b7-4692-ad5f-0d74167bd0d4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:16:27.194164 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ca69d1-88b7-4692-ad5f-0d74167bd0d4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:16:27.194466 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ca69d1-88b7-4692-ad5f-0d74167bd0d4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:16:27.196236 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ca69d1-88b7-4692-ad5f-0d74167bd0d4 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:16:27.209530 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1067/2140] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Jul 25 12:16:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:16:27.213982 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5255b8ed-8aa4-4c76-8616-275c24a6a2d5 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:16:27.223665 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5255b8ed-8aa4-4c76-8616-275c24a6a2d5 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:16:27.228330 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0728fae-2ffb-4dea-9ce2-4d0a762e1aea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:16:27.228984 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0728fae-2ffb-4dea-9ce2-4d0a762e1aea admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:16:27.229268 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0728fae-2ffb-4dea-9ce2-4d0a762e1aea admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:16:27.229548 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0728fae-2ffb-4dea-9ce2-4d0a762e1aea admin admin] PATH_INFO: `/v3/projects` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:16:27.230283 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0728fae-2ffb-4dea-9ce2-4d0a762e1aea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:16:27.232213 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0728fae-2ffb-4dea-9ce2-4d0a762e1aea admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:16:27.258586 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1074/2141] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Jul 25 12:16:27 2022] POST /identity/v3/projects => generated 391 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:16:27.262922 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ca69d1-88b7-4692-ad5f-0d74167bd0d4 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:16:27.273835 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ca69d1-88b7-4692-ad5f-0d74167bd0d4 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:16:27.315053 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:16:27.335417 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6436af6-b27b-4fa3-b27c-708ed42c7a0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:16:27.336120 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6436af6-b27b-4fa3-b27c-708ed42c7a0b admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:16:27.336402 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6436af6-b27b-4fa3-b27c-708ed42c7a0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:16:27.336684 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6436af6-b27b-4fa3-b27c-708ed42c7a0b admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:16:27.337389 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6436af6-b27b-4fa3-b27c-708ed42c7a0b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:16:27.339385 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6436af6-b27b-4fa3-b27c-708ed42c7a0b admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:16:27.361992 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1068/2142] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Jul 25 12:16:27 2022] POST /identity/v3/users => generated 296 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:16:27.366274 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0728fae-2ffb-4dea-9ce2-4d0a762e1aea admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:16:27.377205 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0728fae-2ffb-4dea-9ce2-4d0a762e1aea admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:16:27.383467 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0118593-1311-44fc-84f4-47e9cf4aff64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:16:27.384227 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0118593-1311-44fc-84f4-47e9cf4aff64 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:16:27.384688 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0118593-1311-44fc-84f4-47e9cf4aff64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:16:27.385071 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0118593-1311-44fc-84f4-47e9cf4aff64 admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:16:27.385847 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0118593-1311-44fc-84f4-47e9cf4aff64 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:16:27.387462 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0118593-1311-44fc-84f4-47e9cf4aff64 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:16:27.402798 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1075/2143] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Jul 25 12:16:27 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:16:27.404214 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6436af6-b27b-4fa3-b27c-708ed42c7a0b admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:16:27.415289 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6436af6-b27b-4fa3-b27c-708ed42c7a0b admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:16:27.421052 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71050558-9905-4244-8b1d-4b9b0acba533 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:16:27.422131 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71050558-9905-4244-8b1d-4b9b0acba533 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:16:27.422131 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71050558-9905-4244-8b1d-4b9b0acba533 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:16:27.422305 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71050558-9905-4244-8b1d-4b9b0acba533 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:16:27.423033 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71050558-9905-4244-8b1d-4b9b0acba533 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:16:27.424756 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71050558-9905-4244-8b1d-4b9b0acba533 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:16:27.437786 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1069/2144] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Jul 25 12:16:27 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:16:27.443686 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0118593-1311-44fc-84f4-47e9cf4aff64 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:16:27.455243 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0118593-1311-44fc-84f4-47e9cf4aff64 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:16:27.460868 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49d4a05-6921-4ce5-ba9c-feec02f9af29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:16:27.461752 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49d4a05-6921-4ce5-ba9c-feec02f9af29 admin admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:16:27.462040 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49d4a05-6921-4ce5-ba9c-feec02f9af29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:16:27.462324 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49d4a05-6921-4ce5-ba9c-feec02f9af29 admin admin] PATH_INFO: `/v3/projects/4d257c063ac841d1bb7c27a374923c8b/users/40dc3bd62641421886e231215b06c427/roles/472313ae69174578824df0bed6390744` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:16:27.491042 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49d4a05-6921-4ce5-ba9c-feec02f9af29 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4d257c063ac841d1bb7c27a374923c8b, user_id=40dc3bd62641421886e231215b06c427, role_id=472313ae69174578824df0bed6390744)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:16:27.494555 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49d4a05-6921-4ce5-ba9c-feec02f9af29 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:16:27.514600 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1076/2145] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Jul 25 12:16:27 2022] PUT /identity/v3/projects/4d257c063ac841d1bb7c27a374923c8b/users/40dc3bd62641421886e231215b06c427/roles/472313ae69174578824df0bed6390744 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:16:27.525423 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161f9c3a-01c9-4e41-a018-ccef3fcae5ed None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:16:27.525735 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161f9c3a-01c9-4e41-a018-ccef3fcae5ed None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:16:27.526021 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161f9c3a-01c9-4e41-a018-ccef3fcae5ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:16:27.555821 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-161f9c3a-01c9-4e41-a018-ccef3fcae5ed None None] MFA Rules not processed for user `40dc3bd62641421886e231215b06c427`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:16:27.557073 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-161f9c3a-01c9-4e41-a018-ccef3fcae5ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:16:27.621570 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1070/2146] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:16:27 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:16:27.626228 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49d4a05-6921-4ce5-ba9c-feec02f9af29 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:16:27.658057 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:16:27.690943 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49d4a05-6921-4ce5-ba9c-feec02f9af29 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:16:27.695615 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc9f74e-ec17-4df9-8837-f7440d9547c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:16:27.696275 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc9f74e-ec17-4df9-8837-f7440d9547c1 admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:16:27.696564 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc9f74e-ec17-4df9-8837-f7440d9547c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:16:27.696846 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc9f74e-ec17-4df9-8837-f7440d9547c1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:16:27.697639 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc9f74e-ec17-4df9-8837-f7440d9547c1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:16:27.699915 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc9f74e-ec17-4df9-8837-f7440d9547c1 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:16:27.726611 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1077/2147] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Jul 25 12:16:27 2022] POST /identity/v3/projects => generated 393 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:16:27.730641 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161f9c3a-01c9-4e41-a018-ccef3fcae5ed None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:16:27.741398 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161f9c3a-01c9-4e41-a018-ccef3fcae5ed None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:16:27.799434 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db699db1-c4d1-4eaf-9cf0-3d3b96ffc85c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:16:27.800154 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db699db1-c4d1-4eaf-9cf0-3d3b96ffc85c admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:16:27.800449 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db699db1-c4d1-4eaf-9cf0-3d3b96ffc85c admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:16:27.800730 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db699db1-c4d1-4eaf-9cf0-3d3b96ffc85c admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:16:27.801429 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db699db1-c4d1-4eaf-9cf0-3d3b96ffc85c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:16:27.803338 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db699db1-c4d1-4eaf-9cf0-3d3b96ffc85c admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:16:27.823892 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1071/2148] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Jul 25 12:16:27 2022] POST /identity/v3/users => generated 303 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:16:27.827968 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc9f74e-ec17-4df9-8837-f7440d9547c1 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:16:27.839280 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc9f74e-ec17-4df9-8837-f7440d9547c1 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:16:27.843987 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148a9d9d-0e7f-4c3a-b843-51fd88b5b4b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:16:27.844660 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148a9d9d-0e7f-4c3a-b843-51fd88b5b4b5 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:16:27.844952 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148a9d9d-0e7f-4c3a-b843-51fd88b5b4b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:16:27.845233 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148a9d9d-0e7f-4c3a-b843-51fd88b5b4b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:16:27.845960 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148a9d9d-0e7f-4c3a-b843-51fd88b5b4b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:16:27.848184 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148a9d9d-0e7f-4c3a-b843-51fd88b5b4b5 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:16:27.858770 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1078/2149] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Jul 25 12:16:27 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:16:27.863021 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db699db1-c4d1-4eaf-9cf0-3d3b96ffc85c admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:16:27.873136 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db699db1-c4d1-4eaf-9cf0-3d3b96ffc85c admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:16:27.877801 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde59f09-2c2f-4720-8018-44a8072ed173 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:16:27.878696 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde59f09-2c2f-4720-8018-44a8072ed173 admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:16:27.878983 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde59f09-2c2f-4720-8018-44a8072ed173 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:16:27.879263 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde59f09-2c2f-4720-8018-44a8072ed173 admin admin] PATH_INFO: `/v3/projects/625e8d0e62c24e1591a9e49ec1aa07fd/users/88bb1b0da7ef4af3b30f055bdcbc7741/roles/a7d20885b6d94928a34b3cc5b3a00006` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:16:27.893748 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde59f09-2c2f-4720-8018-44a8072ed173 admin admin] RBAC: Authorizing `identity:create_grant(project_id=625e8d0e62c24e1591a9e49ec1aa07fd, user_id=88bb1b0da7ef4af3b30f055bdcbc7741, role_id=a7d20885b6d94928a34b3cc5b3a00006)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:16:27.897136 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde59f09-2c2f-4720-8018-44a8072ed173 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:16:27.911594 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1072/2150] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Jul 25 12:16:27 2022] PUT /identity/v3/projects/625e8d0e62c24e1591a9e49ec1aa07fd/users/88bb1b0da7ef4af3b30f055bdcbc7741/roles/a7d20885b6d94928a34b3cc5b3a00006 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:16:27.915442 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148a9d9d-0e7f-4c3a-b843-51fd88b5b4b5 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:16:27.925586 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148a9d9d-0e7f-4c3a-b843-51fd88b5b4b5 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:16:27.987463 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439f3555-5e21-420b-89b7-d67cf251f0a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:16:27.988208 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439f3555-5e21-420b-89b7-d67cf251f0a1 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:16:27.988494 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439f3555-5e21-420b-89b7-d67cf251f0a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:16:27.988782 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439f3555-5e21-420b-89b7-d67cf251f0a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:16:27.989522 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439f3555-5e21-420b-89b7-d67cf251f0a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:16:27.994011 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439f3555-5e21-420b-89b7-d67cf251f0a1 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:16:28.014306 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1079/2151] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Jul 25 12:16:27 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:16:28.016036 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde59f09-2c2f-4720-8018-44a8072ed173 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:16:28.027592 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde59f09-2c2f-4720-8018-44a8072ed173 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:16:28.032289 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef597d2f-a9d4-4d40-8ec4-58aeb4a1de57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:16:28.033183 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef597d2f-a9d4-4d40-8ec4-58aeb4a1de57 admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:16:28.033473 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef597d2f-a9d4-4d40-8ec4-58aeb4a1de57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:16:28.034010 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef597d2f-a9d4-4d40-8ec4-58aeb4a1de57 admin admin] PATH_INFO: `/v3/projects/625e8d0e62c24e1591a9e49ec1aa07fd/users/88bb1b0da7ef4af3b30f055bdcbc7741/roles/a7d20885b6d94928a34b3cc5b3a00006` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:16:28.036794 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef597d2f-a9d4-4d40-8ec4-58aeb4a1de57 admin admin] RBAC: Authorizing `identity:create_grant(project_id=625e8d0e62c24e1591a9e49ec1aa07fd, user_id=88bb1b0da7ef4af3b30f055bdcbc7741, role_id=a7d20885b6d94928a34b3cc5b3a00006)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:16:28.039469 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef597d2f-a9d4-4d40-8ec4-58aeb4a1de57 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:16:28.051365 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1073/2152] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Jul 25 12:16:28 2022] PUT /identity/v3/projects/625e8d0e62c24e1591a9e49ec1aa07fd/users/88bb1b0da7ef4af3b30f055bdcbc7741/roles/a7d20885b6d94928a34b3cc5b3a00006 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:16:28.063288 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8458e36-cbca-4646-9455-862708ab8ac7 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:16:28.063619 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8458e36-cbca-4646-9455-862708ab8ac7 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:16:28.063939 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8458e36-cbca-4646-9455-862708ab8ac7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:16:28.098954 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-d8458e36-cbca-4646-9455-862708ab8ac7 None None] MFA Rules not processed for user `88bb1b0da7ef4af3b30f055bdcbc7741`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:16:28.100233 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-d8458e36-cbca-4646-9455-862708ab8ac7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:16:28.174853 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1080/2153] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:16:28 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 12:16:28.186434 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef597d2f-a9d4-4d40-8ec4-58aeb4a1de57 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:16:28.188296 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-ef597d2f-a9d4-4d40-8ec4-58aeb4a1de57 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:16:28.194173 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:16:28.249366 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef597d2f-a9d4-4d40-8ec4-58aeb4a1de57 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:16:28.252965 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95a4288-3f6e-4454-8f37-b65f5a9536f5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f89ce6f0cf04373a414d927fa0f8a7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:16:28.254109 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95a4288-3f6e-4454-8f37-b65f5a9536f5 service cinder] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:16:28.254398 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95a4288-3f6e-4454-8f37-b65f5a9536f5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:16:28.254684 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95a4288-3f6e-4454-8f37-b65f5a9536f5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:16:28.264200 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95a4288-3f6e-4454-8f37-b65f5a9536f5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:16:28.265655 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95a4288-3f6e-4454-8f37-b65f5a9536f5 service cinder] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:16:28.276624 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1074/2154] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:16:28 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:16:28.324644 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8458e36-cbca-4646-9455-862708ab8ac7 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:16:28.326042 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-d8458e36-cbca-4646-9455-862708ab8ac7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:16:28.338148 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8458e36-cbca-4646-9455-862708ab8ac7 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:16:28.341799 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8db3ac-a978-4514-8747-4e8074834d5a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f89ce6f0cf04373a414d927fa0f8a7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:16:28.342441 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8db3ac-a978-4514-8747-4e8074834d5a service cinder] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:16:28.342732 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8db3ac-a978-4514-8747-4e8074834d5a service cinder] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:16:28.343017 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8db3ac-a978-4514-8747-4e8074834d5a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:16:28.363918 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:16:28.407937 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8db3ac-a978-4514-8747-4e8074834d5a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:16:28.410437 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8db3ac-a978-4514-8747-4e8074834d5a service cinder] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:16:28.420949 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1081/2155] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:16:28 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:16:31.888597 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95a4288-3f6e-4454-8f37-b65f5a9536f5 service cinder] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:16:31.948566 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95a4288-3f6e-4454-8f37-b65f5a9536f5 service cinder] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:16:31.953014 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85310d31-60ba-4620-983f-bf15af2b2687 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:16:31.953869 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85310d31-60ba-4620-983f-bf15af2b2687 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:16:31.954150 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85310d31-60ba-4620-983f-bf15af2b2687 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:16:31.954423 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85310d31-60ba-4620-983f-bf15af2b2687 admin admin] PATH_INFO: `/v3/users/40dc3bd62641421886e231215b06c427` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:16:31.955628 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85310d31-60ba-4620-983f-bf15af2b2687 admin admin] RBAC: Authorizing `identity:delete_user(user_id=40dc3bd62641421886e231215b06c427)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:16:31.957563 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85310d31-60ba-4620-983f-bf15af2b2687 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:16:32.003852 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-85310d31-60ba-4620-983f-bf15af2b2687 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '40dc3bd62641421886e231215b06c427'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:16:32.010871 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-85310d31-60ba-4620-983f-bf15af2b2687 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '40dc3bd62641421886e231215b06c427'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:16:32.017809 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-85310d31-60ba-4620-983f-bf15af2b2687 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '40dc3bd62641421886e231215b06c427'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:16:32.029588 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-85310d31-60ba-4620-983f-bf15af2b2687 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '40dc3bd62641421886e231215b06c427'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:16:32.030232 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-85310d31-60ba-4620-983f-bf15af2b2687 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '40dc3bd62641421886e231215b06c427'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:16:32.037010 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1075/2156] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Jul 25 12:16:31 2022] DELETE /identity/v3/users/40dc3bd62641421886e231215b06c427 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:16:32.041455 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54845c2-5439-4317-a021-1cc44450d48d None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:16:32.041932 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54845c2-5439-4317-a021-1cc44450d48d None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:16:32.042039 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54845c2-5439-4317-a021-1cc44450d48d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:16:32.061313 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-d54845c2-5439-4317-a021-1cc44450d48d None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:16:32.062522 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-d54845c2-5439-4317-a021-1cc44450d48d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:16:32.122320 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1082/2157] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:16:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:16:32.132436 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85310d31-60ba-4620-983f-bf15af2b2687 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:16:32.133885 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-85310d31-60ba-4620-983f-bf15af2b2687 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:16:32.165254 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:16:32.202023 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85310d31-60ba-4620-983f-bf15af2b2687 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:16:32.205782 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7588d6-bd69-4368-97da-90c03735f5bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51733781201243e8abf9eca39b207750', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:16:32.206947 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7588d6-bd69-4368-97da-90c03735f5bc service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:16:32.207236 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7588d6-bd69-4368-97da-90c03735f5bc service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:16:32.207520 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7588d6-bd69-4368-97da-90c03735f5bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:16:32.218013 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7588d6-bd69-4368-97da-90c03735f5bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:16:32.219502 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7588d6-bd69-4368-97da-90c03735f5bc service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:16:32.231491 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1076/2158] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:16:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:16:32.643373 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54845c2-5439-4317-a021-1cc44450d48d None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:16:32.645578 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-d54845c2-5439-4317-a021-1cc44450d48d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:16:32.658724 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54845c2-5439-4317-a021-1cc44450d48d None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:16:32.663411 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c1b908-3369-4f2b-9f87-8e7563a33bcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:16:32.664311 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c1b908-3369-4f2b-9f87-8e7563a33bcd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:16:32.664601 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c1b908-3369-4f2b-9f87-8e7563a33bcd admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:16:32.664915 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c1b908-3369-4f2b-9f87-8e7563a33bcd admin admin] PATH_INFO: `/v3/projects/4d257c063ac841d1bb7c27a374923c8b` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:16:32.666121 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c1b908-3369-4f2b-9f87-8e7563a33bcd admin admin] RBAC: Authorizing `identity:delete_project(project_id=4d257c063ac841d1bb7c27a374923c8b)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:16:32.668090 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c1b908-3369-4f2b-9f87-8e7563a33bcd admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:16:32.716040 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-f0c1b908-3369-4f2b-9f87-8e7563a33bcd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4d257c063ac841d1bb7c27a374923c8b', 'request_id': 'req-f0c1b908-3369-4f2b-9f87-8e7563a33bcd'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:16:32.724055 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-f0c1b908-3369-4f2b-9f87-8e7563a33bcd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4d257c063ac841d1bb7c27a374923c8b', 'request_id': 'req-f0c1b908-3369-4f2b-9f87-8e7563a33bcd'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:16:32.729947 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-f0c1b908-3369-4f2b-9f87-8e7563a33bcd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4d257c063ac841d1bb7c27a374923c8b', 'request_id': 'req-f0c1b908-3369-4f2b-9f87-8e7563a33bcd'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:16:32.740799 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-f0c1b908-3369-4f2b-9f87-8e7563a33bcd admin admin] The token cache is being invalidate because project 4d257c063ac841d1bb7c27a374923c8b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80298) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 12:16:32.741092 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-f0c1b908-3369-4f2b-9f87-8e7563a33bcd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:16:32.741733 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-f0c1b908-3369-4f2b-9f87-8e7563a33bcd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:16:32.743312 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1083/2159] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Jul 25 12:16:32 2022] DELETE /identity/v3/projects/4d257c063ac841d1bb7c27a374923c8b => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:16:32.747304 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7588d6-bd69-4368-97da-90c03735f5bc service neutron] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:16:32.757485 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7588d6-bd69-4368-97da-90c03735f5bc service neutron] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:16:32.812991 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67abb6f2-c679-46b8-81f3-a8d6ce3f1490 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:16:32.813858 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67abb6f2-c679-46b8-81f3-a8d6ce3f1490 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:16:32.814145 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67abb6f2-c679-46b8-81f3-a8d6ce3f1490 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:16:32.814425 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67abb6f2-c679-46b8-81f3-a8d6ce3f1490 admin admin] PATH_INFO: `/v3/users/88bb1b0da7ef4af3b30f055bdcbc7741` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:16:32.816165 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67abb6f2-c679-46b8-81f3-a8d6ce3f1490 admin admin] RBAC: Authorizing `identity:delete_user(user_id=88bb1b0da7ef4af3b30f055bdcbc7741)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:16:32.818081 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67abb6f2-c679-46b8-81f3-a8d6ce3f1490 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:16:32.864995 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-67abb6f2-c679-46b8-81f3-a8d6ce3f1490 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '88bb1b0da7ef4af3b30f055bdcbc7741'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:16:32.874132 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-67abb6f2-c679-46b8-81f3-a8d6ce3f1490 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '88bb1b0da7ef4af3b30f055bdcbc7741'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:16:32.882637 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-67abb6f2-c679-46b8-81f3-a8d6ce3f1490 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '88bb1b0da7ef4af3b30f055bdcbc7741'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:16:32.895497 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-67abb6f2-c679-46b8-81f3-a8d6ce3f1490 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '88bb1b0da7ef4af3b30f055bdcbc7741'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:16:32.896182 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-67abb6f2-c679-46b8-81f3-a8d6ce3f1490 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '88bb1b0da7ef4af3b30f055bdcbc7741'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:16:32.902154 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1077/2160] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Jul 25 12:16:32 2022] DELETE /identity/v3/users/88bb1b0da7ef4af3b30f055bdcbc7741 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:16:33.266361 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c1b908-3369-4f2b-9f87-8e7563a33bcd admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:16:33.267988 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-f0c1b908-3369-4f2b-9f87-8e7563a33bcd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:16:33.281562 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c1b908-3369-4f2b-9f87-8e7563a33bcd admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:16:33.347314 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:16:33.366460 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b529d8-ff0a-4f4a-8955-1d3bba14ee91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:16:33.367147 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b529d8-ff0a-4f4a-8955-1d3bba14ee91 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:16:33.368107 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b529d8-ff0a-4f4a-8955-1d3bba14ee91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:16:33.369174 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b529d8-ff0a-4f4a-8955-1d3bba14ee91 admin admin] PATH_INFO: `/v3/projects/625e8d0e62c24e1591a9e49ec1aa07fd` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:16:33.371158 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b529d8-ff0a-4f4a-8955-1d3bba14ee91 admin admin] RBAC: Authorizing `identity:delete_project(project_id=625e8d0e62c24e1591a9e49ec1aa07fd)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:16:33.373458 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b529d8-ff0a-4f4a-8955-1d3bba14ee91 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:16:33.449280 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-21b529d8-ff0a-4f4a-8955-1d3bba14ee91 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '625e8d0e62c24e1591a9e49ec1aa07fd', 'request_id': 'req-21b529d8-ff0a-4f4a-8955-1d3bba14ee91'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:16:33.465646 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-21b529d8-ff0a-4f4a-8955-1d3bba14ee91 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '625e8d0e62c24e1591a9e49ec1aa07fd', 'request_id': 'req-21b529d8-ff0a-4f4a-8955-1d3bba14ee91'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:16:33.485446 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-21b529d8-ff0a-4f4a-8955-1d3bba14ee91 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '625e8d0e62c24e1591a9e49ec1aa07fd', 'request_id': 'req-21b529d8-ff0a-4f4a-8955-1d3bba14ee91'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:16:33.520700 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-21b529d8-ff0a-4f4a-8955-1d3bba14ee91 admin admin] The token cache is being invalidate because project 625e8d0e62c24e1591a9e49ec1aa07fd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80298) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 12:16:33.521010 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-21b529d8-ff0a-4f4a-8955-1d3bba14ee91 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:16:33.524869 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-21b529d8-ff0a-4f4a-8955-1d3bba14ee91 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:16:33.531579 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2315c554-98cd-44e7-be91-883232d07635 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:16:33.531979 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2315c554-98cd-44e7-be91-883232d07635 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:16:33.532283 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2315c554-98cd-44e7-be91-883232d07635 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:16:33.533317 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1084/2161] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Jul 25 12:16:33 2022] DELETE /identity/v3/projects/625e8d0e62c24e1591a9e49ec1aa07fd => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 12:16:33.566038 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-2315c554-98cd-44e7-be91-883232d07635 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:16:33.567765 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-2315c554-98cd-44e7-be91-883232d07635 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:16:33.634165 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:16:33.663761 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1078/2162] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:16:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 12:16:33.665939 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430fae69-1993-472a-a45a-2f60a3c70619 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:16:33.666329 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430fae69-1993-472a-a45a-2f60a3c70619 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:16:33.666711 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430fae69-1993-472a-a45a-2f60a3c70619 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:16:33.690838 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-430fae69-1993-472a-a45a-2f60a3c70619 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:16:33.692274 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-430fae69-1993-472a-a45a-2f60a3c70619 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:16:33.701060 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1085/2163] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:16:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:16:33.705496 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2315c554-98cd-44e7-be91-883232d07635 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:16:33.719681 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2315c554-98cd-44e7-be91-883232d07635 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:16:33.726391 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512ea445-0dfe-48de-a0b8-9204f2b1c8bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:16:33.727170 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512ea445-0dfe-48de-a0b8-9204f2b1c8bd admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:16:33.727471 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512ea445-0dfe-48de-a0b8-9204f2b1c8bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:16:33.729035 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512ea445-0dfe-48de-a0b8-9204f2b1c8bd admin admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:16:33.729035 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512ea445-0dfe-48de-a0b8-9204f2b1c8bd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:16:33.729035 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512ea445-0dfe-48de-a0b8-9204f2b1c8bd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:16:33.730572 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512ea445-0dfe-48de-a0b8-9204f2b1c8bd admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:16:33.748996 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1079/2164] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Jul 25 12:16:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:16:33.750832 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430fae69-1993-472a-a45a-2f60a3c70619 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:16:33.765502 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430fae69-1993-472a-a45a-2f60a3c70619 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:16:33.776702 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e998e79-7733-4f36-b937-ef46d87bf8b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:16:33.777395 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e998e79-7733-4f36-b937-ef46d87bf8b2 admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:16:33.777688 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e998e79-7733-4f36-b937-ef46d87bf8b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:16:33.777971 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e998e79-7733-4f36-b937-ef46d87bf8b2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:16:33.778682 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e998e79-7733-4f36-b937-ef46d87bf8b2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:16:33.780637 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e998e79-7733-4f36-b937-ef46d87bf8b2 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:16:33.817114 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1086/2165] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Jul 25 12:16:33 2022] POST /identity/v3/projects => generated 401 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:16:33.824711 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512ea445-0dfe-48de-a0b8-9204f2b1c8bd admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:16:33.835886 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512ea445-0dfe-48de-a0b8-9204f2b1c8bd admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:16:33.914531 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ccff79-bb52-4052-a022-8fd644ae8f95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:16:33.915365 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ccff79-bb52-4052-a022-8fd644ae8f95 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:16:33.915796 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ccff79-bb52-4052-a022-8fd644ae8f95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:16:33.916174 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ccff79-bb52-4052-a022-8fd644ae8f95 admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:16:33.917008 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ccff79-bb52-4052-a022-8fd644ae8f95 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:16:33.919007 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ccff79-bb52-4052-a022-8fd644ae8f95 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:16:33.942787 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1080/2166] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Jul 25 12:16:33 2022] POST /identity/v3/users => generated 301 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:16:33.947103 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e998e79-7733-4f36-b937-ef46d87bf8b2 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:16:33.959279 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e998e79-7733-4f36-b937-ef46d87bf8b2 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:16:33.964766 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d483665d-16b6-4db8-b9c7-ea8bd42072a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:16:33.965441 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d483665d-16b6-4db8-b9c7-ea8bd42072a9 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:16:33.965734 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d483665d-16b6-4db8-b9c7-ea8bd42072a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:16:33.966026 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d483665d-16b6-4db8-b9c7-ea8bd42072a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:16:33.966995 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d483665d-16b6-4db8-b9c7-ea8bd42072a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:16:33.968640 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d483665d-16b6-4db8-b9c7-ea8bd42072a9 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:16:33.983955 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1087/2167] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Jul 25 12:16:33 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:16:33.986250 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ccff79-bb52-4052-a022-8fd644ae8f95 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:16:33.999636 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ccff79-bb52-4052-a022-8fd644ae8f95 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:16:34.006646 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a07e28-c601-4eb9-aaad-974a9ea0d6a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:16:34.007756 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a07e28-c601-4eb9-aaad-974a9ea0d6a4 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:16:34.008127 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a07e28-c601-4eb9-aaad-974a9ea0d6a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:16:34.008418 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a07e28-c601-4eb9-aaad-974a9ea0d6a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:16:34.009164 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a07e28-c601-4eb9-aaad-974a9ea0d6a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:16:34.010895 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a07e28-c601-4eb9-aaad-974a9ea0d6a4 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:16:34.032861 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d483665d-16b6-4db8-b9c7-ea8bd42072a9 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:16:34.039673 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1081/2168] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Jul 25 12:16:33 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:16:34.050362 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d483665d-16b6-4db8-b9c7-ea8bd42072a9 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:16:34.056827 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012b0a09-c68c-4858-8202-72bcecbb1344 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:16:34.057735 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012b0a09-c68c-4858-8202-72bcecbb1344 admin admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:16:34.058306 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012b0a09-c68c-4858-8202-72bcecbb1344 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:16:34.058594 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012b0a09-c68c-4858-8202-72bcecbb1344 admin admin] PATH_INFO: `/v3/projects/244e0eefabe5494bbd8eb4980e1585a6/users/5bfe1c8978a848ebaa1d0ed2e87f4b44/roles/472313ae69174578824df0bed6390744` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:16:34.079852 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012b0a09-c68c-4858-8202-72bcecbb1344 admin admin] RBAC: Authorizing `identity:create_grant(project_id=244e0eefabe5494bbd8eb4980e1585a6, user_id=5bfe1c8978a848ebaa1d0ed2e87f4b44, role_id=472313ae69174578824df0bed6390744)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:16:34.087078 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012b0a09-c68c-4858-8202-72bcecbb1344 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:16:34.117818 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1088/2169] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Jul 25 12:16:34 2022] PUT /identity/v3/projects/244e0eefabe5494bbd8eb4980e1585a6/users/5bfe1c8978a848ebaa1d0ed2e87f4b44/roles/472313ae69174578824df0bed6390744 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:16:34.120269 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a9cdcd-d9d2-4a2a-82ae-def22ad6f246 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:16:34.120697 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a9cdcd-d9d2-4a2a-82ae-def22ad6f246 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:16:34.121069 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a9cdcd-d9d2-4a2a-82ae-def22ad6f246 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:16:34.163578 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-e8a9cdcd-d9d2-4a2a-82ae-def22ad6f246 None None] MFA Rules not processed for user `5bfe1c8978a848ebaa1d0ed2e87f4b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:16:34.164885 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-e8a9cdcd-d9d2-4a2a-82ae-def22ad6f246 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:16:34.263815 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1082/2170] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:16:34 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:16:34.270153 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012b0a09-c68c-4858-8202-72bcecbb1344 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:16:34.357496 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012b0a09-c68c-4858-8202-72bcecbb1344 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:16:34.362840 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77a9f48-1aa8-4f72-9ba2-5226138fd2dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:16:34.363503 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77a9f48-1aa8-4f72-9ba2-5226138fd2dc admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:16:34.363824 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77a9f48-1aa8-4f72-9ba2-5226138fd2dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:16:34.364110 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77a9f48-1aa8-4f72-9ba2-5226138fd2dc admin admin] PATH_INFO: `/v3/projects` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:16:34.364815 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77a9f48-1aa8-4f72-9ba2-5226138fd2dc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:16:34.366764 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77a9f48-1aa8-4f72-9ba2-5226138fd2dc admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:16:34.387973 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:16:34.400656 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1089/2171] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Jul 25 12:16:34 2022] POST /identity/v3/projects => generated 399 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:16:34.401858 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a9cdcd-d9d2-4a2a-82ae-def22ad6f246 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:16:34.412875 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a9cdcd-d9d2-4a2a-82ae-def22ad6f246 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:16:34.452684 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:16:34.470493 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e5eb78-112d-42e4-a121-95c81a2d26f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:16:34.471155 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e5eb78-112d-42e4-a121-95c81a2d26f3 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:16:34.471443 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e5eb78-112d-42e4-a121-95c81a2d26f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:16:34.471754 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e5eb78-112d-42e4-a121-95c81a2d26f3 admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:16:34.472443 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e5eb78-112d-42e4-a121-95c81a2d26f3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:16:34.474351 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e5eb78-112d-42e4-a121-95c81a2d26f3 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:16:34.495498 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1083/2172] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Jul 25 12:16:34 2022] POST /identity/v3/users => generated 306 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:16:34.500396 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77a9f48-1aa8-4f72-9ba2-5226138fd2dc admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:16:34.513249 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77a9f48-1aa8-4f72-9ba2-5226138fd2dc admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:16:34.523369 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6885f312-be8a-4467-82d8-21c9f3d746cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:16:34.524083 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6885f312-be8a-4467-82d8-21c9f3d746cd admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:16:34.524370 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6885f312-be8a-4467-82d8-21c9f3d746cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:16:34.524696 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6885f312-be8a-4467-82d8-21c9f3d746cd admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:16:34.525493 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6885f312-be8a-4467-82d8-21c9f3d746cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:16:34.527104 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6885f312-be8a-4467-82d8-21c9f3d746cd admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:16:34.538557 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1090/2173] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Jul 25 12:16:34 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:16:34.543037 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e5eb78-112d-42e4-a121-95c81a2d26f3 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:16:34.553554 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e5eb78-112d-42e4-a121-95c81a2d26f3 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:16:34.558228 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a606514e-28fb-498c-bb52-e3d4877a791b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:16:34.559100 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a606514e-28fb-498c-bb52-e3d4877a791b admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:16:34.559381 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a606514e-28fb-498c-bb52-e3d4877a791b admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:16:34.559687 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a606514e-28fb-498c-bb52-e3d4877a791b admin admin] PATH_INFO: `/v3/projects/fe9b908260464bd3b83f29b9e53eea73/users/0be8394f782441739bf79975c2fc27b3/roles/a7d20885b6d94928a34b3cc5b3a00006` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:16:34.575757 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a606514e-28fb-498c-bb52-e3d4877a791b admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe9b908260464bd3b83f29b9e53eea73, user_id=0be8394f782441739bf79975c2fc27b3, role_id=a7d20885b6d94928a34b3cc5b3a00006)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:16:34.578417 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a606514e-28fb-498c-bb52-e3d4877a791b admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:16:34.591097 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1084/2174] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Jul 25 12:16:34 2022] PUT /identity/v3/projects/fe9b908260464bd3b83f29b9e53eea73/users/0be8394f782441739bf79975c2fc27b3/roles/a7d20885b6d94928a34b3cc5b3a00006 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 12:16:34.595204 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6885f312-be8a-4467-82d8-21c9f3d746cd admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:16:34.606435 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6885f312-be8a-4467-82d8-21c9f3d746cd admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:16:34.663818 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c4f714-ea9b-4786-95fb-5fbf9bc6fb9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:16:34.664762 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c4f714-ea9b-4786-95fb-5fbf9bc6fb9a admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:16:34.665066 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c4f714-ea9b-4786-95fb-5fbf9bc6fb9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:16:34.665499 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c4f714-ea9b-4786-95fb-5fbf9bc6fb9a admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:16:34.666343 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c4f714-ea9b-4786-95fb-5fbf9bc6fb9a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:16:34.668297 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c4f714-ea9b-4786-95fb-5fbf9bc6fb9a admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:16:34.679499 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1091/2175] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Jul 25 12:16:34 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:16:34.683845 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a606514e-28fb-498c-bb52-e3d4877a791b admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:16:34.694553 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a606514e-28fb-498c-bb52-e3d4877a791b admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:16:34.699515 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fe5ddc-7826-4ff2-a068-ed08413d400d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:16:34.700414 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fe5ddc-7826-4ff2-a068-ed08413d400d admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:16:34.700697 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fe5ddc-7826-4ff2-a068-ed08413d400d admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:16:34.700971 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fe5ddc-7826-4ff2-a068-ed08413d400d admin admin] PATH_INFO: `/v3/projects/fe9b908260464bd3b83f29b9e53eea73/users/0be8394f782441739bf79975c2fc27b3/roles/a7d20885b6d94928a34b3cc5b3a00006` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:16:34.702898 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86fe5ddc-7826-4ff2-a068-ed08413d400d admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe9b908260464bd3b83f29b9e53eea73, user_id=0be8394f782441739bf79975c2fc27b3, role_id=a7d20885b6d94928a34b3cc5b3a00006)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:16:34.705546 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86fe5ddc-7826-4ff2-a068-ed08413d400d admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:16:34.715472 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1085/2176] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Jul 25 12:16:34 2022] PUT /identity/v3/projects/fe9b908260464bd3b83f29b9e53eea73/users/0be8394f782441739bf79975c2fc27b3/roles/a7d20885b6d94928a34b3cc5b3a00006 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:16:34.726186 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf92ceb-d8fe-4841-9813-3b649e496498 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:16:34.726488 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf92ceb-d8fe-4841-9813-3b649e496498 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:16:34.726915 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf92ceb-d8fe-4841-9813-3b649e496498 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:16:34.756805 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-dcf92ceb-d8fe-4841-9813-3b649e496498 None None] MFA Rules not processed for user `0be8394f782441739bf79975c2fc27b3`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:16:34.757966 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-dcf92ceb-d8fe-4841-9813-3b649e496498 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:16:34.832708 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1092/2177] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:16:34 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:16:34.843257 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fe5ddc-7826-4ff2-a068-ed08413d400d admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:16:34.845090 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-86fe5ddc-7826-4ff2-a068-ed08413d400d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:16:34.905488 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fe5ddc-7826-4ff2-a068-ed08413d400d admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:16:34.909961 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a0b695-138f-4e3a-9d17-cdae705baf8c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f89ce6f0cf04373a414d927fa0f8a7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:16:34.910641 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a0b695-138f-4e3a-9d17-cdae705baf8c service cinder] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:16:34.910928 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a0b695-138f-4e3a-9d17-cdae705baf8c service cinder] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:16:34.911213 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a0b695-138f-4e3a-9d17-cdae705baf8c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:16:34.920558 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a0b695-138f-4e3a-9d17-cdae705baf8c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:16:34.921986 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a0b695-138f-4e3a-9d17-cdae705baf8c service cinder] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:16:34.932256 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1086/2178] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:16:34 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:16:35.008840 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf92ceb-d8fe-4841-9813-3b649e496498 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:16:35.018650 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf92ceb-d8fe-4841-9813-3b649e496498 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:16:35.022262 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3d2694-e2c0-4253-b333-e27460e5452b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f89ce6f0cf04373a414d927fa0f8a7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:16:35.022907 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3d2694-e2c0-4253-b333-e27460e5452b service cinder] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:16:35.023197 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3d2694-e2c0-4253-b333-e27460e5452b service cinder] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:16:35.023482 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3d2694-e2c0-4253-b333-e27460e5452b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:16:35.084232 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3d2694-e2c0-4253-b333-e27460e5452b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:16:35.085661 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3d2694-e2c0-4253-b333-e27460e5452b service cinder] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:16:35.094511 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1093/2179] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:16:35 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:16:39.156375 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a0b695-138f-4e3a-9d17-cdae705baf8c service cinder] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:16:39.219684 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a0b695-138f-4e3a-9d17-cdae705baf8c service cinder] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:16:39.224258 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496ccc41-6851-4339-8bd4-1d6f424556c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:16:39.225126 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496ccc41-6851-4339-8bd4-1d6f424556c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:16:39.225414 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496ccc41-6851-4339-8bd4-1d6f424556c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:16:39.225695 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496ccc41-6851-4339-8bd4-1d6f424556c0 admin admin] PATH_INFO: `/v3/users/5bfe1c8978a848ebaa1d0ed2e87f4b44` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:16:39.226915 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496ccc41-6851-4339-8bd4-1d6f424556c0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5bfe1c8978a848ebaa1d0ed2e87f4b44)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:16:39.228904 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496ccc41-6851-4339-8bd4-1d6f424556c0 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:16:39.270570 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=90,keystone:INSERT=5,keystone:DELETE=5 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:16:39.277634 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-496ccc41-6851-4339-8bd4-1d6f424556c0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5bfe1c8978a848ebaa1d0ed2e87f4b44'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:16:39.285362 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-496ccc41-6851-4339-8bd4-1d6f424556c0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5bfe1c8978a848ebaa1d0ed2e87f4b44'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:16:39.292215 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-496ccc41-6851-4339-8bd4-1d6f424556c0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5bfe1c8978a848ebaa1d0ed2e87f4b44'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:16:39.305654 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-496ccc41-6851-4339-8bd4-1d6f424556c0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5bfe1c8978a848ebaa1d0ed2e87f4b44'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:16:39.306580 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-496ccc41-6851-4339-8bd4-1d6f424556c0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5bfe1c8978a848ebaa1d0ed2e87f4b44'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:16:39.312743 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1087/2180] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Jul 25 12:16:39 2022] DELETE /identity/v3/users/5bfe1c8978a848ebaa1d0ed2e87f4b44 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:16:39.317760 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9abe23-9970-4b02-a11f-bd65fd9edb3d None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:16:39.318245 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9abe23-9970-4b02-a11f-bd65fd9edb3d None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:16:39.318353 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9abe23-9970-4b02-a11f-bd65fd9edb3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:16:39.331873 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:16:39.337676 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-7c9abe23-9970-4b02-a11f-bd65fd9edb3d None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:16:39.338816 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-7c9abe23-9970-4b02-a11f-bd65fd9edb3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:16:39.396695 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1094/2181] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:16:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:16:39.406228 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496ccc41-6851-4339-8bd4-1d6f424556c0 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:16:39.407703 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-496ccc41-6851-4339-8bd4-1d6f424556c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:16:39.464399 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496ccc41-6851-4339-8bd4-1d6f424556c0 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:16:39.468040 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ced0760-d665-4ada-bf2b-8b8d983aa342 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51733781201243e8abf9eca39b207750', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:16:39.468688 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ced0760-d665-4ada-bf2b-8b8d983aa342 service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:16:39.468981 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ced0760-d665-4ada-bf2b-8b8d983aa342 service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:16:39.469269 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ced0760-d665-4ada-bf2b-8b8d983aa342 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:16:39.478578 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ced0760-d665-4ada-bf2b-8b8d983aa342 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:16:39.480063 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ced0760-d665-4ada-bf2b-8b8d983aa342 service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:16:39.489639 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1088/2182] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:16:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:16:39.759794 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9abe23-9970-4b02-a11f-bd65fd9edb3d None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:16:39.761341 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-7c9abe23-9970-4b02-a11f-bd65fd9edb3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:16:39.786703 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9abe23-9970-4b02-a11f-bd65fd9edb3d None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:16:39.791894 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744f9318-be34-4ead-8941-b7d3fe705ee5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:16:39.792771 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744f9318-be34-4ead-8941-b7d3fe705ee5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:16:39.793090 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744f9318-be34-4ead-8941-b7d3fe705ee5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:16:39.793398 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744f9318-be34-4ead-8941-b7d3fe705ee5 admin admin] PATH_INFO: `/v3/projects/244e0eefabe5494bbd8eb4980e1585a6` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:16:39.794597 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744f9318-be34-4ead-8941-b7d3fe705ee5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=244e0eefabe5494bbd8eb4980e1585a6)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:16:39.796594 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744f9318-be34-4ead-8941-b7d3fe705ee5 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:16:39.844658 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-744f9318-be34-4ead-8941-b7d3fe705ee5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '244e0eefabe5494bbd8eb4980e1585a6', 'request_id': 'req-744f9318-be34-4ead-8941-b7d3fe705ee5'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:16:39.851796 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-744f9318-be34-4ead-8941-b7d3fe705ee5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '244e0eefabe5494bbd8eb4980e1585a6', 'request_id': 'req-744f9318-be34-4ead-8941-b7d3fe705ee5'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:16:39.857389 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-744f9318-be34-4ead-8941-b7d3fe705ee5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '244e0eefabe5494bbd8eb4980e1585a6', 'request_id': 'req-744f9318-be34-4ead-8941-b7d3fe705ee5'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:16:39.869261 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-744f9318-be34-4ead-8941-b7d3fe705ee5 admin admin] The token cache is being invalidate because project 244e0eefabe5494bbd8eb4980e1585a6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80298) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 12:16:39.869492 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-744f9318-be34-4ead-8941-b7d3fe705ee5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:16:39.870134 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-744f9318-be34-4ead-8941-b7d3fe705ee5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:16:39.871702 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1095/2183] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Jul 25 12:16:39 2022] DELETE /identity/v3/projects/244e0eefabe5494bbd8eb4980e1585a6 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:16:39.875645 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ced0760-d665-4ada-bf2b-8b8d983aa342 service neutron] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:16:39.886178 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ced0760-d665-4ada-bf2b-8b8d983aa342 service neutron] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:16:39.940902 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9afce2-c619-490e-bb47-e8620f57e572 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:16:39.941772 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9afce2-c619-490e-bb47-e8620f57e572 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:16:39.942065 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9afce2-c619-490e-bb47-e8620f57e572 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:16:39.942350 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9afce2-c619-490e-bb47-e8620f57e572 admin admin] PATH_INFO: `/v3/users/0be8394f782441739bf79975c2fc27b3` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:16:39.943589 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9afce2-c619-490e-bb47-e8620f57e572 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0be8394f782441739bf79975c2fc27b3)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:16:39.945535 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9afce2-c619-490e-bb47-e8620f57e572 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:16:39.992885 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-4c9afce2-c619-490e-bb47-e8620f57e572 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0be8394f782441739bf79975c2fc27b3'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:16:39.999717 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-4c9afce2-c619-490e-bb47-e8620f57e572 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0be8394f782441739bf79975c2fc27b3'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:16:40.003223 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:16:40.007143 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-4c9afce2-c619-490e-bb47-e8620f57e572 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0be8394f782441739bf79975c2fc27b3'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:16:40.021001 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-4c9afce2-c619-490e-bb47-e8620f57e572 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0be8394f782441739bf79975c2fc27b3'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:16:40.021656 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-4c9afce2-c619-490e-bb47-e8620f57e572 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0be8394f782441739bf79975c2fc27b3'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:16:40.027600 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1089/2184] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Jul 25 12:16:39 2022] DELETE /identity/v3/users/0be8394f782441739bf79975c2fc27b3 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:16:40.313102 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744f9318-be34-4ead-8941-b7d3fe705ee5 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:16:40.314674 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-744f9318-be34-4ead-8941-b7d3fe705ee5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:16:40.340566 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744f9318-be34-4ead-8941-b7d3fe705ee5 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:16:40.403251 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5420cd-d022-4b5c-850b-af38971cdf01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:16:40.404102 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5420cd-d022-4b5c-850b-af38971cdf01 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:16:40.404388 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5420cd-d022-4b5c-850b-af38971cdf01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:16:40.404667 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5420cd-d022-4b5c-850b-af38971cdf01 admin admin] PATH_INFO: `/v3/projects/fe9b908260464bd3b83f29b9e53eea73` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:16:40.405874 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5420cd-d022-4b5c-850b-af38971cdf01 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fe9b908260464bd3b83f29b9e53eea73)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:16:40.407818 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5420cd-d022-4b5c-850b-af38971cdf01 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:16:40.445291 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:16:40.461374 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-9b5420cd-d022-4b5c-850b-af38971cdf01 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fe9b908260464bd3b83f29b9e53eea73', 'request_id': 'req-9b5420cd-d022-4b5c-850b-af38971cdf01'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:16:40.469334 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-9b5420cd-d022-4b5c-850b-af38971cdf01 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fe9b908260464bd3b83f29b9e53eea73', 'request_id': 'req-9b5420cd-d022-4b5c-850b-af38971cdf01'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:16:40.476453 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-9b5420cd-d022-4b5c-850b-af38971cdf01 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fe9b908260464bd3b83f29b9e53eea73', 'request_id': 'req-9b5420cd-d022-4b5c-850b-af38971cdf01'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:16:40.487979 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-9b5420cd-d022-4b5c-850b-af38971cdf01 admin admin] The token cache is being invalidate because project fe9b908260464bd3b83f29b9e53eea73 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80298) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 12:16:40.489931 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-9b5420cd-d022-4b5c-850b-af38971cdf01 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:16:40.490627 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-9b5420cd-d022-4b5c-850b-af38971cdf01 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:16:40.492229 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1096/2185] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Jul 25 12:16:40 2022] DELETE /identity/v3/projects/fe9b908260464bd3b83f29b9e53eea73 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:16:40.499759 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73758e07-3e71-483f-a6b6-b7a02d82142f None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:16:40.500258 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73758e07-3e71-483f-a6b6-b7a02d82142f None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:16:40.500355 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73758e07-3e71-483f-a6b6-b7a02d82142f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:16:40.520885 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-73758e07-3e71-483f-a6b6-b7a02d82142f None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:16:40.522090 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-73758e07-3e71-483f-a6b6-b7a02d82142f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:16:40.582070 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1090/2186] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:16:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:16:40.589754 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425e3bc9-e6fa-48a6-ab2d-0021616d11eb None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:16:40.590046 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425e3bc9-e6fa-48a6-ab2d-0021616d11eb None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:16:40.590324 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425e3bc9-e6fa-48a6-ab2d-0021616d11eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:16:40.611036 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-425e3bc9-e6fa-48a6-ab2d-0021616d11eb None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:16:40.612263 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-425e3bc9-e6fa-48a6-ab2d-0021616d11eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:16:40.620561 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1097/2187] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:16:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:16:40.624729 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73758e07-3e71-483f-a6b6-b7a02d82142f None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:16:40.634301 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73758e07-3e71-483f-a6b6-b7a02d82142f None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:16:40.639129 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0b3512-1be6-493c-a49f-b76c94f36bd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:16:40.639825 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0b3512-1be6-493c-a49f-b76c94f36bd8 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:16:40.640117 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0b3512-1be6-493c-a49f-b76c94f36bd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:16:40.640419 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0b3512-1be6-493c-a49f-b76c94f36bd8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:16:40.641141 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0b3512-1be6-493c-a49f-b76c94f36bd8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:16:40.641456 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0b3512-1be6-493c-a49f-b76c94f36bd8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:16:40.643102 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0b3512-1be6-493c-a49f-b76c94f36bd8 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:16:40.655124 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1091/2188] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Jul 25 12:16:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:16:40.659944 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425e3bc9-e6fa-48a6-ab2d-0021616d11eb None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:16:40.669082 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425e3bc9-e6fa-48a6-ab2d-0021616d11eb None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:16:40.673856 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df07209-c1f2-46bd-a110-92494247f426 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:16:40.674502 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df07209-c1f2-46bd-a110-92494247f426 admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:16:40.674794 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df07209-c1f2-46bd-a110-92494247f426 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:16:40.675079 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df07209-c1f2-46bd-a110-92494247f426 admin admin] PATH_INFO: `/v3/projects` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:16:40.675843 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df07209-c1f2-46bd-a110-92494247f426 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:16:40.677744 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df07209-c1f2-46bd-a110-92494247f426 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:16:40.706610 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1098/2189] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Jul 25 12:16:40 2022] POST /identity/v3/projects => generated 373 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:16:40.710493 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0b3512-1be6-493c-a49f-b76c94f36bd8 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:16:40.720649 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0b3512-1be6-493c-a49f-b76c94f36bd8 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:16:40.777299 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55fef00-6c8f-48c7-bcc0-1cfe676e63e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:16:40.777974 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55fef00-6c8f-48c7-bcc0-1cfe676e63e1 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:16:40.778257 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55fef00-6c8f-48c7-bcc0-1cfe676e63e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:16:40.778529 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55fef00-6c8f-48c7-bcc0-1cfe676e63e1 admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:16:40.779228 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55fef00-6c8f-48c7-bcc0-1cfe676e63e1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:16:40.781243 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55fef00-6c8f-48c7-bcc0-1cfe676e63e1 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:16:40.803850 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1092/2190] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Jul 25 12:16:40 2022] POST /identity/v3/users => generated 287 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:16:40.807949 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df07209-c1f2-46bd-a110-92494247f426 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:16:40.819636 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df07209-c1f2-46bd-a110-92494247f426 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:16:40.828705 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d127a9-4fa7-49ba-b28d-7b18da166660 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:16:40.829671 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d127a9-4fa7-49ba-b28d-7b18da166660 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:16:40.830138 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d127a9-4fa7-49ba-b28d-7b18da166660 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:16:40.830574 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d127a9-4fa7-49ba-b28d-7b18da166660 admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:16:40.832089 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d127a9-4fa7-49ba-b28d-7b18da166660 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:16:40.834455 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d127a9-4fa7-49ba-b28d-7b18da166660 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:16:40.845339 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1099/2191] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Jul 25 12:16:40 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:16:40.849060 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55fef00-6c8f-48c7-bcc0-1cfe676e63e1 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:16:40.859324 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55fef00-6c8f-48c7-bcc0-1cfe676e63e1 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:16:40.864471 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686a84f0-b298-4693-94aa-3c8a25794335 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:16:40.865247 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686a84f0-b298-4693-94aa-3c8a25794335 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:16:40.865545 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686a84f0-b298-4693-94aa-3c8a25794335 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:16:40.865823 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686a84f0-b298-4693-94aa-3c8a25794335 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:16:40.866560 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686a84f0-b298-4693-94aa-3c8a25794335 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:16:40.868214 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686a84f0-b298-4693-94aa-3c8a25794335 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:16:40.880591 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1093/2192] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Jul 25 12:16:40 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:16:40.884361 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d127a9-4fa7-49ba-b28d-7b18da166660 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:16:40.895538 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d127a9-4fa7-49ba-b28d-7b18da166660 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:16:40.900225 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cd5970-c78b-44ae-aaed-86656be00c17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:16:40.901093 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cd5970-c78b-44ae-aaed-86656be00c17 admin admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:16:40.901374 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cd5970-c78b-44ae-aaed-86656be00c17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:16:40.901649 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cd5970-c78b-44ae-aaed-86656be00c17 admin admin] PATH_INFO: `/v3/projects/0576c7e1636d41bdb67f739fcbaedcd8/users/f8fd582ce5ae48d6a21354d19970bb54/roles/472313ae69174578824df0bed6390744` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:16:40.918441 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84cd5970-c78b-44ae-aaed-86656be00c17 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0576c7e1636d41bdb67f739fcbaedcd8, user_id=f8fd582ce5ae48d6a21354d19970bb54, role_id=472313ae69174578824df0bed6390744)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:16:40.921066 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84cd5970-c78b-44ae-aaed-86656be00c17 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:16:40.935088 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1100/2193] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Jul 25 12:16:40 2022] PUT /identity/v3/projects/0576c7e1636d41bdb67f739fcbaedcd8/users/f8fd582ce5ae48d6a21354d19970bb54/roles/472313ae69174578824df0bed6390744 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:16:40.947144 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d6f8c1-b553-4121-97c4-39a1984085e2 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:16:40.947440 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d6f8c1-b553-4121-97c4-39a1984085e2 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:16:40.947764 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d6f8c1-b553-4121-97c4-39a1984085e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:16:40.975430 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-d7d6f8c1-b553-4121-97c4-39a1984085e2 None None] MFA Rules not processed for user `f8fd582ce5ae48d6a21354d19970bb54`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:16:40.976651 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-d7d6f8c1-b553-4121-97c4-39a1984085e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:16:41.007777 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:16:41.041558 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1094/2194] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:16:40 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:16:41.053145 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cd5970-c78b-44ae-aaed-86656be00c17 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:16:41.054879 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-84cd5970-c78b-44ae-aaed-86656be00c17 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:16:41.111439 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cd5970-c78b-44ae-aaed-86656be00c17 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:16:41.115823 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bba7b27-ec17-47f2-adfc-4da2721121e7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f89ce6f0cf04373a414d927fa0f8a7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:16:41.116548 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bba7b27-ec17-47f2-adfc-4da2721121e7 service cinder] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:16:41.116909 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bba7b27-ec17-47f2-adfc-4da2721121e7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:16:41.117261 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bba7b27-ec17-47f2-adfc-4da2721121e7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:16:41.127036 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bba7b27-ec17-47f2-adfc-4da2721121e7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:16:41.129402 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bba7b27-ec17-47f2-adfc-4da2721121e7 service cinder] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:16:41.138770 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1101/2195] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:16:41 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:16:41.163651 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d6f8c1-b553-4121-97c4-39a1984085e2 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:16:41.231963 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d6f8c1-b553-4121-97c4-39a1984085e2 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:16:41.238244 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fe6779-8f70-4fe5-ae5d-0c69c12ea2fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:16:41.239236 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fe6779-8f70-4fe5-ae5d-0c69c12ea2fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:16:41.239653 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fe6779-8f70-4fe5-ae5d-0c69c12ea2fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:16:41.240056 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fe6779-8f70-4fe5-ae5d-0c69c12ea2fb admin admin] PATH_INFO: `/v3/users/f8fd582ce5ae48d6a21354d19970bb54` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:16:41.241386 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64fe6779-8f70-4fe5-ae5d-0c69c12ea2fb admin admin] RBAC: Authorizing `identity:delete_user(user_id=f8fd582ce5ae48d6a21354d19970bb54)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:16:41.243712 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64fe6779-8f70-4fe5-ae5d-0c69c12ea2fb admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:16:41.300086 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-64fe6779-8f70-4fe5-ae5d-0c69c12ea2fb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f8fd582ce5ae48d6a21354d19970bb54'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:16:41.307105 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-64fe6779-8f70-4fe5-ae5d-0c69c12ea2fb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f8fd582ce5ae48d6a21354d19970bb54'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:16:41.314659 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-64fe6779-8f70-4fe5-ae5d-0c69c12ea2fb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f8fd582ce5ae48d6a21354d19970bb54'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:16:41.327910 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-64fe6779-8f70-4fe5-ae5d-0c69c12ea2fb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f8fd582ce5ae48d6a21354d19970bb54'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:16:41.328628 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-64fe6779-8f70-4fe5-ae5d-0c69c12ea2fb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f8fd582ce5ae48d6a21354d19970bb54'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:16:41.334743 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1095/2196] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Jul 25 12:16:41 2022] DELETE /identity/v3/users/f8fd582ce5ae48d6a21354d19970bb54 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:16:41.339206 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90472e2b-e0d0-46f3-b5e3-0ed664cd6eb2 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:16:41.339504 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90472e2b-e0d0-46f3-b5e3-0ed664cd6eb2 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:16:41.339856 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90472e2b-e0d0-46f3-b5e3-0ed664cd6eb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:16:41.358978 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-90472e2b-e0d0-46f3-b5e3-0ed664cd6eb2 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:16:41.360228 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-90472e2b-e0d0-46f3-b5e3-0ed664cd6eb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:16:41.421322 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:16:41.432585 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1102/2197] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:16:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:16:41.442879 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fe6779-8f70-4fe5-ae5d-0c69c12ea2fb admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:16:41.444395 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-64fe6779-8f70-4fe5-ae5d-0c69c12ea2fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:16:41.509503 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fe6779-8f70-4fe5-ae5d-0c69c12ea2fb admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:16:41.513476 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984e6b45-81d2-4e77-ad32-9ec61d80ccba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51733781201243e8abf9eca39b207750', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:16:41.514148 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984e6b45-81d2-4e77-ad32-9ec61d80ccba service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:16:41.514438 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984e6b45-81d2-4e77-ad32-9ec61d80ccba service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:16:41.514723 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984e6b45-81d2-4e77-ad32-9ec61d80ccba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:16:41.524085 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:16:41.528537 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984e6b45-81d2-4e77-ad32-9ec61d80ccba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:16:41.530079 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984e6b45-81d2-4e77-ad32-9ec61d80ccba service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:16:41.551980 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1096/2198] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:16:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:16:42.127974 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90472e2b-e0d0-46f3-b5e3-0ed664cd6eb2 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:16:42.129424 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-90472e2b-e0d0-46f3-b5e3-0ed664cd6eb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:16:42.141290 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90472e2b-e0d0-46f3-b5e3-0ed664cd6eb2 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:16:42.155821 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331ead42-fb08-46d1-ab7b-1343a4896150 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:16:42.156719 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331ead42-fb08-46d1-ab7b-1343a4896150 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:16:42.157015 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331ead42-fb08-46d1-ab7b-1343a4896150 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:16:42.157298 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331ead42-fb08-46d1-ab7b-1343a4896150 admin admin] PATH_INFO: `/v3/projects/0576c7e1636d41bdb67f739fcbaedcd8` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:16:42.159363 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331ead42-fb08-46d1-ab7b-1343a4896150 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0576c7e1636d41bdb67f739fcbaedcd8)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:16:42.162171 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331ead42-fb08-46d1-ab7b-1343a4896150 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:16:42.218529 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-331ead42-fb08-46d1-ab7b-1343a4896150 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0576c7e1636d41bdb67f739fcbaedcd8', 'request_id': 'req-331ead42-fb08-46d1-ab7b-1343a4896150'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:16:42.226216 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-331ead42-fb08-46d1-ab7b-1343a4896150 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0576c7e1636d41bdb67f739fcbaedcd8', 'request_id': 'req-331ead42-fb08-46d1-ab7b-1343a4896150'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:16:42.231803 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-331ead42-fb08-46d1-ab7b-1343a4896150 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0576c7e1636d41bdb67f739fcbaedcd8', 'request_id': 'req-331ead42-fb08-46d1-ab7b-1343a4896150'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:16:42.242665 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-331ead42-fb08-46d1-ab7b-1343a4896150 admin admin] The token cache is being invalidate because project 0576c7e1636d41bdb67f739fcbaedcd8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80298) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 12:16:42.242952 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-331ead42-fb08-46d1-ab7b-1343a4896150 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:16:42.243603 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-331ead42-fb08-46d1-ab7b-1343a4896150 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:16:42.245404 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1103/2199] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Jul 25 12:16:42 2022] DELETE /identity/v3/projects/0576c7e1636d41bdb67f739fcbaedcd8 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:16:42.249975 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95818e6-5f19-41d3-8977-c9385661d610 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:16:42.250271 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95818e6-5f19-41d3-8977-c9385661d610 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:16:42.250559 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95818e6-5f19-41d3-8977-c9385661d610 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:16:42.270840 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-a95818e6-5f19-41d3-8977-c9385661d610 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:16:42.272072 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-a95818e6-5f19-41d3-8977-c9385661d610 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:16:42.330326 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1097/2200] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:16:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 12:16:42.338480 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046433bc-978c-4450-9734-cb8f219a1753 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:16:42.338782 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046433bc-978c-4450-9734-cb8f219a1753 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:16:42.339065 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046433bc-978c-4450-9734-cb8f219a1753 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:16:42.358135 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-046433bc-978c-4450-9734-cb8f219a1753 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:16:42.359317 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-046433bc-978c-4450-9734-cb8f219a1753 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:16:42.367145 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1104/2201] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:16:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:16:42.371069 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95818e6-5f19-41d3-8977-c9385661d610 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:16:42.381235 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95818e6-5f19-41d3-8977-c9385661d610 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:16:42.385806 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4f0d43-d0e8-44ce-b2c7-a98011c6afec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:16:42.386458 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4f0d43-d0e8-44ce-b2c7-a98011c6afec admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:16:42.386744 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4f0d43-d0e8-44ce-b2c7-a98011c6afec admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:16:42.387025 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4f0d43-d0e8-44ce-b2c7-a98011c6afec admin admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:16:42.387795 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4f0d43-d0e8-44ce-b2c7-a98011c6afec admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:16:42.388105 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4f0d43-d0e8-44ce-b2c7-a98011c6afec admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:16:42.389693 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4f0d43-d0e8-44ce-b2c7-a98011c6afec admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:16:42.401802 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1098/2202] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Jul 25 12:16:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 12:16:42.405882 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046433bc-978c-4450-9734-cb8f219a1753 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:16:42.415397 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046433bc-978c-4450-9734-cb8f219a1753 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:16:42.420083 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62756eea-2ed2-4a88-bb39-5981e6ae6ba8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:16:42.420728 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62756eea-2ed2-4a88-bb39-5981e6ae6ba8 admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:16:42.421014 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62756eea-2ed2-4a88-bb39-5981e6ae6ba8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:16:42.421293 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62756eea-2ed2-4a88-bb39-5981e6ae6ba8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:16:42.421990 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62756eea-2ed2-4a88-bb39-5981e6ae6ba8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:16:42.423911 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62756eea-2ed2-4a88-bb39-5981e6ae6ba8 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:16:42.452201 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1105/2203] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Jul 25 12:16:42 2022] POST /identity/v3/projects => generated 375 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 12:16:42.457352 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4f0d43-d0e8-44ce-b2c7-a98011c6afec admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:16:42.470323 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4f0d43-d0e8-44ce-b2c7-a98011c6afec admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:16:42.525961 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda80b7c-5e8e-4720-a979-5d52411d1c6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:16:42.526660 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda80b7c-5e8e-4720-a979-5d52411d1c6f admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:16:42.526951 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda80b7c-5e8e-4720-a979-5d52411d1c6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:16:42.527236 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda80b7c-5e8e-4720-a979-5d52411d1c6f admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:16:42.527982 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda80b7c-5e8e-4720-a979-5d52411d1c6f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:16:42.530002 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda80b7c-5e8e-4720-a979-5d52411d1c6f admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:16:42.552488 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1099/2204] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Jul 25 12:16:42 2022] POST /identity/v3/users => generated 288 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:16:42.556762 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62756eea-2ed2-4a88-bb39-5981e6ae6ba8 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:16:42.569346 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62756eea-2ed2-4a88-bb39-5981e6ae6ba8 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:16:42.574851 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5782e1-04a7-463f-8431-23a187671848 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:16:42.575911 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5782e1-04a7-463f-8431-23a187671848 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:16:42.576538 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5782e1-04a7-463f-8431-23a187671848 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:16:42.576818 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5782e1-04a7-463f-8431-23a187671848 admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:16:42.577564 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5782e1-04a7-463f-8431-23a187671848 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:16:42.579238 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5782e1-04a7-463f-8431-23a187671848 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:16:42.594440 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1106/2205] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Jul 25 12:16:42 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 12:16:42.595659 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda80b7c-5e8e-4720-a979-5d52411d1c6f admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:16:42.605940 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda80b7c-5e8e-4720-a979-5d52411d1c6f admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:16:42.610631 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ddc5e0-6315-4266-ac5b-99718439ac98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:16:42.611363 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ddc5e0-6315-4266-ac5b-99718439ac98 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:16:42.611688 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ddc5e0-6315-4266-ac5b-99718439ac98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:16:42.612485 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ddc5e0-6315-4266-ac5b-99718439ac98 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:16:42.613213 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ddc5e0-6315-4266-ac5b-99718439ac98 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:16:42.614813 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ddc5e0-6315-4266-ac5b-99718439ac98 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:16:42.626705 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1100/2206] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Jul 25 12:16:42 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 12:16:42.631419 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5782e1-04a7-463f-8431-23a187671848 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:16:42.641508 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5782e1-04a7-463f-8431-23a187671848 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:16:42.646393 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b6324d-32ad-4a3a-9ca9-54d76c2966bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:16:42.647263 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b6324d-32ad-4a3a-9ca9-54d76c2966bd admin admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:16:42.647628 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b6324d-32ad-4a3a-9ca9-54d76c2966bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:16:42.647868 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b6324d-32ad-4a3a-9ca9-54d76c2966bd admin admin] PATH_INFO: `/v3/projects/e6aa467128894d90942eb7c49511a111/users/67e77dd6574f4928a099c38538bb00a2/roles/472313ae69174578824df0bed6390744` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:16:42.663319 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b6324d-32ad-4a3a-9ca9-54d76c2966bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=e6aa467128894d90942eb7c49511a111, user_id=67e77dd6574f4928a099c38538bb00a2, role_id=472313ae69174578824df0bed6390744)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:16:42.666096 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b6324d-32ad-4a3a-9ca9-54d76c2966bd admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:16:42.680806 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1107/2207] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Jul 25 12:16:42 2022] PUT /identity/v3/projects/e6aa467128894d90942eb7c49511a111/users/67e77dd6574f4928a099c38538bb00a2/roles/472313ae69174578824df0bed6390744 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:16:42.691694 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d16eae0-7191-4e52-b57d-7eb54d8e5a52 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:16:42.691993 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d16eae0-7191-4e52-b57d-7eb54d8e5a52 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:16:42.692275 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d16eae0-7191-4e52-b57d-7eb54d8e5a52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:16:42.720222 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-1d16eae0-7191-4e52-b57d-7eb54d8e5a52 None None] MFA Rules not processed for user `67e77dd6574f4928a099c38538bb00a2`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:16:42.721435 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-1d16eae0-7191-4e52-b57d-7eb54d8e5a52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:16:42.773189 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:16:42.795497 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1101/2208] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:16:42 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:16:42.806798 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b6324d-32ad-4a3a-9ca9-54d76c2966bd admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:16:42.808523 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-c4b6324d-32ad-4a3a-9ca9-54d76c2966bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:16:42.863786 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b6324d-32ad-4a3a-9ca9-54d76c2966bd admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:16:42.867729 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b89c480-387e-4490-897a-efe55d1a5d52 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f89ce6f0cf04373a414d927fa0f8a7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:16:42.868380 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b89c480-387e-4490-897a-efe55d1a5d52 service cinder] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:16:42.868674 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b89c480-387e-4490-897a-efe55d1a5d52 service cinder] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:16:42.868962 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b89c480-387e-4490-897a-efe55d1a5d52 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:16:42.878316 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b89c480-387e-4490-897a-efe55d1a5d52 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:16:42.879850 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b89c480-387e-4490-897a-efe55d1a5d52 service cinder] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:16:42.889645 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1108/2209] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:16:42 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:16:45.070503 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d16eae0-7191-4e52-b57d-7eb54d8e5a52 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:16:45.071956 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-1d16eae0-7191-4e52-b57d-7eb54d8e5a52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:16:45.129477 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d16eae0-7191-4e52-b57d-7eb54d8e5a52 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:16:45.133041 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512eac50-efdd-4560-b57d-75189ed3d31e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51733781201243e8abf9eca39b207750', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:16:45.133718 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512eac50-efdd-4560-b57d-75189ed3d31e service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:16:45.133964 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512eac50-efdd-4560-b57d-75189ed3d31e service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:16:45.134253 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512eac50-efdd-4560-b57d-75189ed3d31e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:16:45.136046 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-512eac50-efdd-4560-b57d-75189ed3d31e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:16:45.200522 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512eac50-efdd-4560-b57d-75189ed3d31e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:16:45.201969 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512eac50-efdd-4560-b57d-75189ed3d31e service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:16:45.211607 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1102/2210] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:16:45 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:16:49.591573 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4520d5eb-dc49-43d3-9c84-ede48b4d5b56 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:16:49.593163 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4520d5eb-dc49-43d3-9c84-ede48b4d5b56 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:16:49.593618 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4520d5eb-dc49-43d3-9c84-ede48b4d5b56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:16:49.631216 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-4520d5eb-dc49-43d3-9c84-ede48b4d5b56 None None] MFA Rules not processed for user `93207b858d294d049fa56d45810a40aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:16:49.633570 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-4520d5eb-dc49-43d3-9c84-ede48b4d5b56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:16:49.654121 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:16:49.725434 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1109/2211] 10.222.0.42 () {58 vars in 1009 bytes} [Mon Jul 25 12:16:49 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:16:49.743112 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512eac50-efdd-4560-b57d-75189ed3d31e service neutron] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:16:49.744589 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-512eac50-efdd-4560-b57d-75189ed3d31e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:16:49.860988 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512eac50-efdd-4560-b57d-75189ed3d31e service neutron] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:16:49.867868 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9150af57-aebf-4b12-843a-abb7b0337c30 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bd782ea66364eeb96592096eae63d4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:16:49.868862 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9150af57-aebf-4b12-843a-abb7b0337c30 service swift] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:16:49.869313 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9150af57-aebf-4b12-843a-abb7b0337c30 service swift] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:16:49.869774 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9150af57-aebf-4b12-843a-abb7b0337c30 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:16:49.881908 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9150af57-aebf-4b12-843a-abb7b0337c30 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:16:49.883515 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9150af57-aebf-4b12-843a-abb7b0337c30 service swift] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:16:49.897696 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1103/2212] 10.222.0.42 () {60 vars in 1384 bytes} [Mon Jul 25 12:16:49 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 25 12:16:56.170681 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4520d5eb-dc49-43d3-9c84-ede48b4d5b56 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:16:56.281362 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4520d5eb-dc49-43d3-9c84-ede48b4d5b56 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:16:56.294905 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc29f72d-ac6e-4a37-9a70-df13018b4b31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:16:56.295802 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc29f72d-ac6e-4a37-9a70-df13018b4b31 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:16:56.296102 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc29f72d-ac6e-4a37-9a70-df13018b4b31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:16:56.296380 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc29f72d-ac6e-4a37-9a70-df13018b4b31 admin admin] PATH_INFO: `/v3/users/67e77dd6574f4928a099c38538bb00a2` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:16:56.297785 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc29f72d-ac6e-4a37-9a70-df13018b4b31 admin admin] RBAC: Authorizing `identity:delete_user(user_id=67e77dd6574f4928a099c38538bb00a2)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:16:56.300386 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc29f72d-ac6e-4a37-9a70-df13018b4b31 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:16:56.422187 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-cc29f72d-ac6e-4a37-9a70-df13018b4b31 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '67e77dd6574f4928a099c38538bb00a2'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:16:56.422187 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-cc29f72d-ac6e-4a37-9a70-df13018b4b31 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '67e77dd6574f4928a099c38538bb00a2'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:16:56.439709 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-cc29f72d-ac6e-4a37-9a70-df13018b4b31 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '67e77dd6574f4928a099c38538bb00a2'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:16:56.497298 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-cc29f72d-ac6e-4a37-9a70-df13018b4b31 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '67e77dd6574f4928a099c38538bb00a2'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:16:56.556157 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-cc29f72d-ac6e-4a37-9a70-df13018b4b31 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '67e77dd6574f4928a099c38538bb00a2'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:16:56.563763 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1110/2213] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Jul 25 12:16:56 2022] DELETE /identity/v3/users/67e77dd6574f4928a099c38538bb00a2 => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:16:56.569041 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4000cec5-e313-4afe-a8dd-174559900563 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:16:56.569970 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4000cec5-e313-4afe-a8dd-174559900563 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:16:56.570481 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4000cec5-e313-4afe-a8dd-174559900563 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:16:56.597867 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-4000cec5-e313-4afe-a8dd-174559900563 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:16:56.605414 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-4000cec5-e313-4afe-a8dd-174559900563 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:16:56.646451 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:16:56.683212 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1104/2214] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:16:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:16:56.692698 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc29f72d-ac6e-4a37-9a70-df13018b4b31 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:16:56.694141 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-cc29f72d-ac6e-4a37-9a70-df13018b4b31 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:16:56.756470 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:16:56.762985 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc29f72d-ac6e-4a37-9a70-df13018b4b31 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:16:56.782201 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc3faa7-ff40-499a-bea5-9e5483695049 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51733781201243e8abf9eca39b207750', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:16:56.783762 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc3faa7-ff40-499a-bea5-9e5483695049 service neutron] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:16:56.783762 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc3faa7-ff40-499a-bea5-9e5483695049 service neutron] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:16:56.783762 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc3faa7-ff40-499a-bea5-9e5483695049 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:16:56.797446 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc3faa7-ff40-499a-bea5-9e5483695049 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:16:56.799522 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc3faa7-ff40-499a-bea5-9e5483695049 service neutron] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:16:56.811840 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1111/2215] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:16:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:16:57.224609 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4000cec5-e313-4afe-a8dd-174559900563 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:16:57.226607 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-4000cec5-e313-4afe-a8dd-174559900563 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:16:57.248938 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4000cec5-e313-4afe-a8dd-174559900563 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:16:57.257099 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395e5622-bc01-4a11-9881-0b043e1d33ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:16:57.258028 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395e5622-bc01-4a11-9881-0b043e1d33ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:16:57.258375 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395e5622-bc01-4a11-9881-0b043e1d33ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:16:57.271488 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395e5622-bc01-4a11-9881-0b043e1d33ea admin admin] PATH_INFO: `/v3/projects/e6aa467128894d90942eb7c49511a111` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:16:57.273150 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395e5622-bc01-4a11-9881-0b043e1d33ea admin admin] RBAC: Authorizing `identity:delete_project(project_id=e6aa467128894d90942eb7c49511a111)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:16:57.275442 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395e5622-bc01-4a11-9881-0b043e1d33ea admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:16:57.357818 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-395e5622-bc01-4a11-9881-0b043e1d33ea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e6aa467128894d90942eb7c49511a111', 'request_id': 'req-395e5622-bc01-4a11-9881-0b043e1d33ea'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:16:57.371797 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-395e5622-bc01-4a11-9881-0b043e1d33ea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e6aa467128894d90942eb7c49511a111', 'request_id': 'req-395e5622-bc01-4a11-9881-0b043e1d33ea'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:16:57.384410 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-395e5622-bc01-4a11-9881-0b043e1d33ea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e6aa467128894d90942eb7c49511a111', 'request_id': 'req-395e5622-bc01-4a11-9881-0b043e1d33ea'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:16:57.403617 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-395e5622-bc01-4a11-9881-0b043e1d33ea admin admin] The token cache is being invalidate because project e6aa467128894d90942eb7c49511a111 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 12:16:57.404009 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-395e5622-bc01-4a11-9881-0b043e1d33ea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:16:57.404696 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-395e5622-bc01-4a11-9881-0b043e1d33ea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:16:57.407589 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1105/2216] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Jul 25 12:16:57 2022] DELETE /identity/v3/projects/e6aa467128894d90942eb7c49511a111 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:16:57.411852 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913c92a2-3efb-4fb5-a919-2aff56f3327e None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:16:57.412341 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913c92a2-3efb-4fb5-a919-2aff56f3327e None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:16:57.412459 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913c92a2-3efb-4fb5-a919-2aff56f3327e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:16:57.434670 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-913c92a2-3efb-4fb5-a919-2aff56f3327e None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:16:57.435987 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-913c92a2-3efb-4fb5-a919-2aff56f3327e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:16:57.504731 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1112/2217] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:16:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:16:57.576206 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82986ea0-5e35-4238-a3fe-3b0e75a21e70 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:16:57.576750 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82986ea0-5e35-4238-a3fe-3b0e75a21e70 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:16:57.576750 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82986ea0-5e35-4238-a3fe-3b0e75a21e70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:16:57.601039 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-82986ea0-5e35-4238-a3fe-3b0e75a21e70 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:16:57.602585 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-82986ea0-5e35-4238-a3fe-3b0e75a21e70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:16:57.615582 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1106/2218] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:16:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:16:57.619976 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913c92a2-3efb-4fb5-a919-2aff56f3327e None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:16:57.631093 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913c92a2-3efb-4fb5-a919-2aff56f3327e None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:16:57.639641 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cc11d1-bdb2-4b8a-9705-3428e9e6db77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:16:57.640479 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cc11d1-bdb2-4b8a-9705-3428e9e6db77 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:16:57.640771 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cc11d1-bdb2-4b8a-9705-3428e9e6db77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:16:57.641058 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cc11d1-bdb2-4b8a-9705-3428e9e6db77 admin admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:16:57.641879 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1cc11d1-bdb2-4b8a-9705-3428e9e6db77 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:16:57.642224 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1cc11d1-bdb2-4b8a-9705-3428e9e6db77 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:16:57.643873 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1cc11d1-bdb2-4b8a-9705-3428e9e6db77 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:16:57.660138 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1113/2219] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Jul 25 12:16:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:16:57.664382 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82986ea0-5e35-4238-a3fe-3b0e75a21e70 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:16:57.680571 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82986ea0-5e35-4238-a3fe-3b0e75a21e70 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:16:57.692426 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e2abbf-472b-4c55-b294-32ac55631baa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:16:57.693742 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e2abbf-472b-4c55-b294-32ac55631baa admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:16:57.693742 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e2abbf-472b-4c55-b294-32ac55631baa admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:16:57.693957 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e2abbf-472b-4c55-b294-32ac55631baa admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:16:57.699528 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e2abbf-472b-4c55-b294-32ac55631baa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:16:57.701730 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e2abbf-472b-4c55-b294-32ac55631baa admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:16:57.739270 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1107/2220] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Jul 25 12:16:57 2022] POST /identity/v3/projects => generated 399 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:16:57.741000 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cc11d1-bdb2-4b8a-9705-3428e9e6db77 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:16:57.756866 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cc11d1-bdb2-4b8a-9705-3428e9e6db77 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:16:57.850045 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482e28c3-c2cf-485c-b55d-a2761eb96558 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:16:57.850830 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482e28c3-c2cf-485c-b55d-a2761eb96558 admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:16:57.851118 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482e28c3-c2cf-485c-b55d-a2761eb96558 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:16:57.851448 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482e28c3-c2cf-485c-b55d-a2761eb96558 admin admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:16:57.852713 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482e28c3-c2cf-485c-b55d-a2761eb96558 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:16:57.854810 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482e28c3-c2cf-485c-b55d-a2761eb96558 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:16:57.883321 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1114/2221] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Jul 25 12:16:57 2022] POST /identity/v3/users => generated 300 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:16:57.887767 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e2abbf-472b-4c55-b294-32ac55631baa admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:16:57.905665 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e2abbf-472b-4c55-b294-32ac55631baa admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:16:57.911848 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269353c5-c6ae-4bc6-8cda-e01872c48b75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:16:57.912518 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269353c5-c6ae-4bc6-8cda-e01872c48b75 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:16:57.912818 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269353c5-c6ae-4bc6-8cda-e01872c48b75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:16:57.913083 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269353c5-c6ae-4bc6-8cda-e01872c48b75 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:16:57.913991 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269353c5-c6ae-4bc6-8cda-e01872c48b75 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:16:57.915850 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269353c5-c6ae-4bc6-8cda-e01872c48b75 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:16:57.928695 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1108/2222] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Jul 25 12:16:57 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 12:16:57.932914 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482e28c3-c2cf-485c-b55d-a2761eb96558 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:16:57.949727 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482e28c3-c2cf-485c-b55d-a2761eb96558 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:16:57.955685 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941bf1ce-9bcd-4583-ad3b-8eec2179ce1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:16:57.956606 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941bf1ce-9bcd-4583-ad3b-8eec2179ce1a admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:16:57.956945 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941bf1ce-9bcd-4583-ad3b-8eec2179ce1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:16:57.957246 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941bf1ce-9bcd-4583-ad3b-8eec2179ce1a admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:16:57.957971 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941bf1ce-9bcd-4583-ad3b-8eec2179ce1a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:16:57.959604 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941bf1ce-9bcd-4583-ad3b-8eec2179ce1a admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:16:57.972551 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1115/2223] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Jul 25 12:16:57 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 12:16:57.977245 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269353c5-c6ae-4bc6-8cda-e01872c48b75 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:16:57.990053 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269353c5-c6ae-4bc6-8cda-e01872c48b75 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:16:57.996090 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291213c2-6834-4108-8f56-329cd2f0eede admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:16:57.997350 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291213c2-6834-4108-8f56-329cd2f0eede admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:16:57.997751 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291213c2-6834-4108-8f56-329cd2f0eede admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:16:57.998231 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291213c2-6834-4108-8f56-329cd2f0eede admin admin] PATH_INFO: `/v3/projects/c4ee80016cc74735b19b40261a20993b/users/dfca8e34759f463da453ce916b69390a/roles/472313ae69174578824df0bed6390744` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:16:58.015867 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291213c2-6834-4108-8f56-329cd2f0eede admin admin] RBAC: Authorizing `identity:create_grant(project_id=c4ee80016cc74735b19b40261a20993b, user_id=dfca8e34759f463da453ce916b69390a, role_id=472313ae69174578824df0bed6390744)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:16:58.019272 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291213c2-6834-4108-8f56-329cd2f0eede admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:16:58.046189 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1109/2224] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Jul 25 12:16:57 2022] PUT /identity/v3/projects/c4ee80016cc74735b19b40261a20993b/users/dfca8e34759f463da453ce916b69390a/roles/472313ae69174578824df0bed6390744 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:16:58.047918 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96be43c2-4105-4956-9840-f2a260a52867 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:16:58.048222 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96be43c2-4105-4956-9840-f2a260a52867 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:16:58.048547 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96be43c2-4105-4956-9840-f2a260a52867 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:16:58.081009 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-96be43c2-4105-4956-9840-f2a260a52867 None None] MFA Rules not processed for user `dfca8e34759f463da453ce916b69390a`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:16:58.082285 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-96be43c2-4105-4956-9840-f2a260a52867 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:16:58.132923 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:16:58.182085 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1116/2225] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:16:58 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:16:58.193169 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291213c2-6834-4108-8f56-329cd2f0eede admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:16:58.196057 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-291213c2-6834-4108-8f56-329cd2f0eede admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:16:58.307322 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291213c2-6834-4108-8f56-329cd2f0eede admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:16:58.324346 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167f6698-dd85-4111-bfce-a89925161058 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f89ce6f0cf04373a414d927fa0f8a7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:16:58.330541 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167f6698-dd85-4111-bfce-a89925161058 service cinder] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:16:58.330796 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167f6698-dd85-4111-bfce-a89925161058 service cinder] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:16:58.331097 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167f6698-dd85-4111-bfce-a89925161058 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:16:58.351842 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:16:58.380566 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167f6698-dd85-4111-bfce-a89925161058 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:16:58.382386 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167f6698-dd85-4111-bfce-a89925161058 service cinder] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:16:58.414377 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1110/2226] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:16:58 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:17:06.927846 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96be43c2-4105-4956-9840-f2a260a52867 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:17:07.039998 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96be43c2-4105-4956-9840-f2a260a52867 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:17:07.046357 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c37d5f-529e-4dc6-bcad-b3b978372224 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:17:07.048880 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c37d5f-529e-4dc6-bcad-b3b978372224 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:17:07.049204 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c37d5f-529e-4dc6-bcad-b3b978372224 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:17:07.049468 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c37d5f-529e-4dc6-bcad-b3b978372224 admin admin] PATH_INFO: `/v3/users/dfca8e34759f463da453ce916b69390a` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:17:07.052567 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c37d5f-529e-4dc6-bcad-b3b978372224 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dfca8e34759f463da453ce916b69390a)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:17:07.056269 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c37d5f-529e-4dc6-bcad-b3b978372224 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:17:07.114366 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-e2c37d5f-529e-4dc6-bcad-b3b978372224 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dfca8e34759f463da453ce916b69390a'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:17:07.115035 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-e2c37d5f-529e-4dc6-bcad-b3b978372224 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dfca8e34759f463da453ce916b69390a'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:17:07.125476 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-e2c37d5f-529e-4dc6-bcad-b3b978372224 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dfca8e34759f463da453ce916b69390a'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:17:07.139048 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-e2c37d5f-529e-4dc6-bcad-b3b978372224 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dfca8e34759f463da453ce916b69390a'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:17:07.146560 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-e2c37d5f-529e-4dc6-bcad-b3b978372224 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dfca8e34759f463da453ce916b69390a'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:17:07.154034 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1117/2227] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Jul 25 12:17:06 2022] DELETE /identity/v3/users/dfca8e34759f463da453ce916b69390a => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:17:07.159778 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8253b911-9167-41fc-9860-464932b274cc None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:17:07.160574 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8253b911-9167-41fc-9860-464932b274cc None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:17:07.160574 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8253b911-9167-41fc-9860-464932b274cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:17:07.182227 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-8253b911-9167-41fc-9860-464932b274cc None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:17:07.183661 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-8253b911-9167-41fc-9860-464932b274cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:17:07.249861 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1111/2228] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:17:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:17:07.260810 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c37d5f-529e-4dc6-bcad-b3b978372224 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:17:07.262445 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-e2c37d5f-529e-4dc6-bcad-b3b978372224 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:17:07.343943 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c37d5f-529e-4dc6-bcad-b3b978372224 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:17:07.351631 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe0cf13-6203-442c-8798-0aaf60172c3f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51733781201243e8abf9eca39b207750', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:17:07.351631 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe0cf13-6203-442c-8798-0aaf60172c3f service neutron] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:17:07.351631 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe0cf13-6203-442c-8798-0aaf60172c3f service neutron] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:17:07.351631 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe0cf13-6203-442c-8798-0aaf60172c3f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:17:07.362950 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe0cf13-6203-442c-8798-0aaf60172c3f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:17:07.364529 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe0cf13-6203-442c-8798-0aaf60172c3f service neutron] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:17:07.375017 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1118/2229] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:17:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:17:07.967351 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8253b911-9167-41fc-9860-464932b274cc None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:17:07.969290 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-8253b911-9167-41fc-9860-464932b274cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:17:07.990375 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8253b911-9167-41fc-9860-464932b274cc None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:17:07.995913 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c7ad38-d9fb-41d0-955a-e4eb579c3fe4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:17:07.996882 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c7ad38-d9fb-41d0-955a-e4eb579c3fe4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:17:07.997121 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c7ad38-d9fb-41d0-955a-e4eb579c3fe4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:17:07.997400 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c7ad38-d9fb-41d0-955a-e4eb579c3fe4 admin admin] PATH_INFO: `/v3/projects/c4ee80016cc74735b19b40261a20993b` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:17:07.998659 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c7ad38-d9fb-41d0-955a-e4eb579c3fe4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c4ee80016cc74735b19b40261a20993b)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:17:08.002892 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c7ad38-d9fb-41d0-955a-e4eb579c3fe4 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:17:08.091207 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-19c7ad38-d9fb-41d0-955a-e4eb579c3fe4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c4ee80016cc74735b19b40261a20993b', 'request_id': 'req-19c7ad38-d9fb-41d0-955a-e4eb579c3fe4'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:17:08.102232 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-19c7ad38-d9fb-41d0-955a-e4eb579c3fe4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c4ee80016cc74735b19b40261a20993b', 'request_id': 'req-19c7ad38-d9fb-41d0-955a-e4eb579c3fe4'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:17:08.113527 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-19c7ad38-d9fb-41d0-955a-e4eb579c3fe4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c4ee80016cc74735b19b40261a20993b', 'request_id': 'req-19c7ad38-d9fb-41d0-955a-e4eb579c3fe4'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:17:08.126759 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-19c7ad38-d9fb-41d0-955a-e4eb579c3fe4 admin admin] The token cache is being invalidate because project c4ee80016cc74735b19b40261a20993b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 12:17:08.127125 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-19c7ad38-d9fb-41d0-955a-e4eb579c3fe4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:17:08.127873 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-19c7ad38-d9fb-41d0-955a-e4eb579c3fe4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:17:08.129854 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1112/2230] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Jul 25 12:17:07 2022] DELETE /identity/v3/projects/c4ee80016cc74735b19b40261a20993b => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:17:08.717862 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d04dee8-d727-4d4b-b533-1b53702b7660 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:17:08.718391 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d04dee8-d727-4d4b-b533-1b53702b7660 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:17:08.718483 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d04dee8-d727-4d4b-b533-1b53702b7660 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:17:08.732025 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:17:08.741912 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-9d04dee8-d727-4d4b-b533-1b53702b7660 None None] MFA Rules not processed for user `93207b858d294d049fa56d45810a40aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:17:08.744468 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-9d04dee8-d727-4d4b-b533-1b53702b7660 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:17:08.811299 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1119/2231] 10.222.0.42 () {58 vars in 967 bytes} [Mon Jul 25 12:17:08 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:17:08.828002 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c7ad38-d9fb-41d0-955a-e4eb579c3fe4 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:17:08.829456 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-19c7ad38-d9fb-41d0-955a-e4eb579c3fe4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:17:08.897374 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c7ad38-d9fb-41d0-955a-e4eb579c3fe4 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:17:08.902431 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc22a3e-53ef-4ff4-97b1-3ac1893fb023 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bd782ea66364eeb96592096eae63d4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:17:08.903075 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc22a3e-53ef-4ff4-97b1-3ac1893fb023 service swift] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:17:08.903364 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc22a3e-53ef-4ff4-97b1-3ac1893fb023 service swift] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:17:08.903678 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc22a3e-53ef-4ff4-97b1-3ac1893fb023 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:17:08.912289 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc22a3e-53ef-4ff4-97b1-3ac1893fb023 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:17:08.913879 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc22a3e-53ef-4ff4-97b1-3ac1893fb023 service swift] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:17:08.922331 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1113/2232] 10.222.0.42 () {60 vars in 1384 bytes} [Mon Jul 25 12:17:08 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 25 12:17:11.005362 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4cb3b6-fcd0-4910-b608-32ed3f0c4e3c None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:17:11.006375 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4cb3b6-fcd0-4910-b608-32ed3f0c4e3c None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:17:11.006836 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4cb3b6-fcd0-4910-b608-32ed3f0c4e3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:17:11.023673 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d3037c-0441-4acd-bf5d-fa64474bcf60 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:17:11.024326 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d3037c-0441-4acd-bf5d-fa64474bcf60 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:17:11.024682 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d3037c-0441-4acd-bf5d-fa64474bcf60 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:17:11.026628 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1114/2233] 10.222.0.42 () {54 vars in 921 bytes} [Mon Jul 25 12:17:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:17:11.034667 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-5c4cb3b6-fcd0-4910-b608-32ed3f0c4e3c None None] MFA Rules not processed for user `a766d17b116f498fab07efb69c94e49e`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:17:11.036559 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-5c4cb3b6-fcd0-4910-b608-32ed3f0c4e3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:17:11.058518 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6c8300-07f5-4a73-9d63-cc69957748c0 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:17:11.058920 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6c8300-07f5-4a73-9d63-cc69957748c0 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:17:11.059325 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6c8300-07f5-4a73-9d63-cc69957748c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:17:11.091740 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:17:11.106931 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-4b6c8300-07f5-4a73-9d63-cc69957748c0 None None] MFA Rules not processed for user `8012afab6cac4358ab878732e58d0689`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:17:11.113108 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-4b6c8300-07f5-4a73-9d63-cc69957748c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:17:11.126267 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1120/2234] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:17:11 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:17:11.127322 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4cb3b6-fcd0-4910-b608-32ed3f0c4e3c None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:17:11.129310 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-5c4cb3b6-fcd0-4910-b608-32ed3f0c4e3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:17:11.236819 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4cb3b6-fcd0-4910-b608-32ed3f0c4e3c None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:17:11.241821 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0aa8ae2-10af-4392-8dcc-9dff43499baa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:17:11.242822 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0aa8ae2-10af-4392-8dcc-9dff43499baa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:17:11.243110 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0aa8ae2-10af-4392-8dcc-9dff43499baa admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:17:11.243395 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0aa8ae2-10af-4392-8dcc-9dff43499baa admin admin] PATH_INFO: `/v3/users/5f696b65c8094119af332e7e837b888e` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:17:11.244679 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0aa8ae2-10af-4392-8dcc-9dff43499baa admin admin] RBAC: Authorizing `identity:delete_user(user_id=5f696b65c8094119af332e7e837b888e)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:17:11.245441 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1115/2235] 10.222.0.42 () {58 vars in 1018 bytes} [Mon Jul 25 12:17:11 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 12:17:11.247023 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0aa8ae2-10af-4392-8dcc-9dff43499baa admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:17:11.267129 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6c8300-07f5-4a73-9d63-cc69957748c0 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:17:11.269764 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-4b6c8300-07f5-4a73-9d63-cc69957748c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:17:11.296080 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6c8300-07f5-4a73-9d63-cc69957748c0 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:17:11.304585 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156af8d7-4223-49c0-b4ad-247d6c38ef15 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8012afab6cac4358ab878732e58d0689', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:17:11.305258 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156af8d7-4223-49c0-b4ad-247d6c38ef15 service placement] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:17:11.305543 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156af8d7-4223-49c0-b4ad-247d6c38ef15 service placement] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:17:11.305828 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156af8d7-4223-49c0-b4ad-247d6c38ef15 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:17:11.314972 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156af8d7-4223-49c0-b4ad-247d6c38ef15 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:17:11.316815 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156af8d7-4223-49c0-b4ad-247d6c38ef15 service placement] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:17:11.317998 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-b0aa8ae2-10af-4392-8dcc-9dff43499baa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5f696b65c8094119af332e7e837b888e'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:17:11.318714 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-b0aa8ae2-10af-4392-8dcc-9dff43499baa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5f696b65c8094119af332e7e837b888e'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:17:11.330956 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-b0aa8ae2-10af-4392-8dcc-9dff43499baa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5f696b65c8094119af332e7e837b888e'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:17:11.338121 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:17:11.340687 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1116/2236] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:17:11 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 75 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:17:11.349923 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-b0aa8ae2-10af-4392-8dcc-9dff43499baa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5f696b65c8094119af332e7e837b888e'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:17:11.359368 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-b0aa8ae2-10af-4392-8dcc-9dff43499baa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5f696b65c8094119af332e7e837b888e'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:17:11.366927 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1121/2237] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Jul 25 12:17:11 2022] DELETE /identity/v3/users/5f696b65c8094119af332e7e837b888e => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:17:11.444831 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019aac88-f7a8-4a0b-9837-d140b211ab1f None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:17:11.445171 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019aac88-f7a8-4a0b-9837-d140b211ab1f None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:17:11.445441 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019aac88-f7a8-4a0b-9837-d140b211ab1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:17:11.470387 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-019aac88-f7a8-4a0b-9837-d140b211ab1f None None] MFA Rules not processed for user `3848eae5b8df48eb9816ba0d7485f7c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:17:11.471867 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-019aac88-f7a8-4a0b-9837-d140b211ab1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:17:11.554492 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0aa8ae2-10af-4392-8dcc-9dff43499baa admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:17:11.556498 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-b0aa8ae2-10af-4392-8dcc-9dff43499baa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:17:11.564789 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1117/2238] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:17:11 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:17:11.644800 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0aa8ae2-10af-4392-8dcc-9dff43499baa admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:17:11.651353 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4469e7-4484-4149-96de-0876d6d5e1c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:17:11.652374 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4469e7-4484-4149-96de-0876d6d5e1c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:17:11.652562 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4469e7-4484-4149-96de-0876d6d5e1c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:17:11.652824 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4469e7-4484-4149-96de-0876d6d5e1c2 admin admin] PATH_INFO: `/v3/projects/91bd04f2c9d34640ab6317fc8dd3dddb` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:17:11.654099 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4469e7-4484-4149-96de-0876d6d5e1c2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=91bd04f2c9d34640ab6317fc8dd3dddb)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:17:11.656079 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4469e7-4484-4149-96de-0876d6d5e1c2 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:17:11.726195 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-2a4469e7-4484-4149-96de-0876d6d5e1c2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '91bd04f2c9d34640ab6317fc8dd3dddb', 'request_id': 'req-2a4469e7-4484-4149-96de-0876d6d5e1c2'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:17:11.735041 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-2a4469e7-4484-4149-96de-0876d6d5e1c2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '91bd04f2c9d34640ab6317fc8dd3dddb', 'request_id': 'req-2a4469e7-4484-4149-96de-0876d6d5e1c2'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:17:11.742645 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-2a4469e7-4484-4149-96de-0876d6d5e1c2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '91bd04f2c9d34640ab6317fc8dd3dddb', 'request_id': 'req-2a4469e7-4484-4149-96de-0876d6d5e1c2'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:17:11.754167 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-2a4469e7-4484-4149-96de-0876d6d5e1c2 admin admin] The token cache is being invalidate because project 91bd04f2c9d34640ab6317fc8dd3dddb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80298) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 12:17:11.754461 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-2a4469e7-4484-4149-96de-0876d6d5e1c2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:17:11.755130 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-2a4469e7-4484-4149-96de-0876d6d5e1c2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:17:11.756868 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1122/2239] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Jul 25 12:17:11 2022] DELETE /identity/v3/projects/91bd04f2c9d34640ab6317fc8dd3dddb => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:17:12.798970 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5ffe38-f120-4a74-96d0-65112768ebf2 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:17:12.799468 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5ffe38-f120-4a74-96d0-65112768ebf2 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:17:12.799745 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5ffe38-f120-4a74-96d0-65112768ebf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:17:12.821573 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-6f5ffe38-f120-4a74-96d0-65112768ebf2 None None] MFA Rules not processed for user `93207b858d294d049fa56d45810a40aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:17:12.822992 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-6f5ffe38-f120-4a74-96d0-65112768ebf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:17:12.891055 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1118/2240] 10.222.0.42 () {58 vars in 1009 bytes} [Mon Jul 25 12:17:12 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:17:12.903799 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4469e7-4484-4149-96de-0876d6d5e1c2 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:17:12.905227 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-2a4469e7-4484-4149-96de-0876d6d5e1c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:17:12.974837 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4469e7-4484-4149-96de-0876d6d5e1c2 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:17:12.979747 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9118a62a-ca06-4e0f-a890-2e0baf467259 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bd782ea66364eeb96592096eae63d4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:17:12.980396 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9118a62a-ca06-4e0f-a890-2e0baf467259 service swift] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:17:12.980688 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9118a62a-ca06-4e0f-a890-2e0baf467259 service swift] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:17:12.980968 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9118a62a-ca06-4e0f-a890-2e0baf467259 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:17:12.988069 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:17:12.990643 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9118a62a-ca06-4e0f-a890-2e0baf467259 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:17:12.993224 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9118a62a-ca06-4e0f-a890-2e0baf467259 service swift] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:17:13.002435 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1123/2241] 10.222.0.42 () {60 vars in 1384 bytes} [Mon Jul 25 12:17:12 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 25 12:17:22.883809 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=82 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:17:23.000772 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=2 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:17:31.184437 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996c1823-4882-4477-8135-2d67d672ff96 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:17:31.185007 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996c1823-4882-4477-8135-2d67d672ff96 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:17:31.185203 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996c1823-4882-4477-8135-2d67d672ff96 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:17:31.186683 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1119/2242] 10.222.0.42 () {54 vars in 921 bytes} [Mon Jul 25 12:17:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:17:31.192852 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b29169d-6755-43ea-b222-5045d6888cb2 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:17:31.193332 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b29169d-6755-43ea-b222-5045d6888cb2 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:17:31.193574 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b29169d-6755-43ea-b222-5045d6888cb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:17:31.214843 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-9b29169d-6755-43ea-b222-5045d6888cb2 None None] MFA Rules not processed for user `f67142a1193d47c7bd33d7040268233c`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:17:31.216105 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-9b29169d-6755-43ea-b222-5045d6888cb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:17:31.283820 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1124/2243] 10.222.0.42 () {58 vars in 1018 bytes} [Mon Jul 25 12:17:31 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 12:17:31.294953 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996c1823-4882-4477-8135-2d67d672ff96 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:17:31.296880 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-996c1823-4882-4477-8135-2d67d672ff96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:17:31.357117 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996c1823-4882-4477-8135-2d67d672ff96 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:17:31.360643 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa1b128-d493-44ff-88af-ee4ec117ce21 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ea16528b1374bae9cf54603f7907bfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:17:31.361292 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa1b128-d493-44ff-88af-ee4ec117ce21 service glance] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:17:31.361582 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa1b128-d493-44ff-88af-ee4ec117ce21 service glance] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:17:31.361858 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa1b128-d493-44ff-88af-ee4ec117ce21 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:17:31.372082 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa1b128-d493-44ff-88af-ee4ec117ce21 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:17:31.373522 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa1b128-d493-44ff-88af-ee4ec117ce21 service glance] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:17:31.384507 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1120/2244] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:17:31 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:17:31.457244 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c923bb1d-e1bd-4077-b0c5-df32d9d35b5f None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:17:31.457539 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c923bb1d-e1bd-4077-b0c5-df32d9d35b5f None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:17:31.457821 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c923bb1d-e1bd-4077-b0c5-df32d9d35b5f None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:17:31.460324 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1125/2245] 10.222.0.42 () {54 vars in 921 bytes} [Mon Jul 25 12:17:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:17:31.465376 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebaeabc4-e406-4d8b-9633-c4a78f147a24 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:17:31.465834 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebaeabc4-e406-4d8b-9633-c4a78f147a24 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:17:31.466181 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebaeabc4-e406-4d8b-9633-c4a78f147a24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:17:31.488882 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-ebaeabc4-e406-4d8b-9633-c4a78f147a24 None None] MFA Rules not processed for user `51733781201243e8abf9eca39b207750`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:17:31.490214 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-ebaeabc4-e406-4d8b-9633-c4a78f147a24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:17:31.548725 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1121/2246] 10.222.0.42 () {58 vars in 1018 bytes} [Mon Jul 25 12:17:31 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 12:17:31.560517 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c923bb1d-e1bd-4077-b0c5-df32d9d35b5f None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:17:31.562720 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-c923bb1d-e1bd-4077-b0c5-df32d9d35b5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:17:31.576788 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c923bb1d-e1bd-4077-b0c5-df32d9d35b5f None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:17:31.580622 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009d663f-772b-4625-93dd-1a655f0b7eff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51733781201243e8abf9eca39b207750', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:17:31.581078 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009d663f-772b-4625-93dd-1a655f0b7eff service neutron] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:17:31.581361 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009d663f-772b-4625-93dd-1a655f0b7eff service neutron] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:17:31.581632 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009d663f-772b-4625-93dd-1a655f0b7eff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:17:31.589511 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009d663f-772b-4625-93dd-1a655f0b7eff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:17:31.591181 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009d663f-772b-4625-93dd-1a655f0b7eff service neutron] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:17:31.599608 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1126/2247] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:17:31 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:17:31.684751 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494e462b-b16d-487c-8a68-5442db65fa14 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:17:31.685040 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494e462b-b16d-487c-8a68-5442db65fa14 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:17:31.685322 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494e462b-b16d-487c-8a68-5442db65fa14 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:17:31.686696 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1122/2248] 10.222.0.42 () {54 vars in 921 bytes} [Mon Jul 25 12:17:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:17:31.691759 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d272809f-04cf-4016-bbe4-5e536f41001c None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:17:31.692081 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d272809f-04cf-4016-bbe4-5e536f41001c None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:17:31.692365 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d272809f-04cf-4016-bbe4-5e536f41001c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:17:31.712093 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-d272809f-04cf-4016-bbe4-5e536f41001c None None] MFA Rules not processed for user `f67142a1193d47c7bd33d7040268233c`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:17:31.713329 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-d272809f-04cf-4016-bbe4-5e536f41001c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:17:31.723581 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1127/2249] 10.222.0.42 () {58 vars in 1018 bytes} [Mon Jul 25 12:17:31 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 12:17:31.733982 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494e462b-b16d-487c-8a68-5442db65fa14 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:17:31.746245 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494e462b-b16d-487c-8a68-5442db65fa14 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:17:31.750008 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5745f2-6bc5-4edc-8019-c212fae9086d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51733781201243e8abf9eca39b207750', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:17:31.750682 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5745f2-6bc5-4edc-8019-c212fae9086d service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:17:31.750974 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5745f2-6bc5-4edc-8019-c212fae9086d service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:17:31.751259 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5745f2-6bc5-4edc-8019-c212fae9086d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:17:31.762176 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5745f2-6bc5-4edc-8019-c212fae9086d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:17:31.763729 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5745f2-6bc5-4edc-8019-c212fae9086d service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:17:31.780922 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1123/2250] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:17:31 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:17:34.563509 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f1133e-a9e2-412a-aed0-4707011ee946 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:17:34.563963 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f1133e-a9e2-412a-aed0-4707011ee946 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:17:34.564134 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f1133e-a9e2-412a-aed0-4707011ee946 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:17:34.582786 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-b6f1133e-a9e2-412a-aed0-4707011ee946 None None] MFA Rules not processed for user `93207b858d294d049fa56d45810a40aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:17:34.584051 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-b6f1133e-a9e2-412a-aed0-4707011ee946 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:17:34.591246 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1128/2251] 10.222.0.42 () {58 vars in 1009 bytes} [Mon Jul 25 12:17:34 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:17:34.602931 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5745f2-6bc5-4edc-8019-c212fae9086d service neutron] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:17:34.615374 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5745f2-6bc5-4edc-8019-c212fae9086d service neutron] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:17:34.625570 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31666b55-257e-442d-8b97-2f23fa19f99b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bd782ea66364eeb96592096eae63d4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:17:34.626233 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31666b55-257e-442d-8b97-2f23fa19f99b service swift] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:17:34.626522 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31666b55-257e-442d-8b97-2f23fa19f99b service swift] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:17:34.626804 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31666b55-257e-442d-8b97-2f23fa19f99b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:17:34.636215 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31666b55-257e-442d-8b97-2f23fa19f99b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:17:34.637811 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31666b55-257e-442d-8b97-2f23fa19f99b service swift] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:17:34.645862 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1124/2252] 10.222.0.42 () {60 vars in 1384 bytes} [Mon Jul 25 12:17:34 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 25 12:17:44.577655 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=43 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:17:44.642678 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=61 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:17:47.852199 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f1133e-a9e2-412a-aed0-4707011ee946 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:17:47.853603 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-b6f1133e-a9e2-412a-aed0-4707011ee946 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:17:47.936508 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f1133e-a9e2-412a-aed0-4707011ee946 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:17:47.941598 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8f6eed-4e24-42bd-9c51-ae49ba72b9dd service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8012afab6cac4358ab878732e58d0689', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:17:47.942261 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8f6eed-4e24-42bd-9c51-ae49ba72b9dd service placement] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:17:47.942551 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8f6eed-4e24-42bd-9c51-ae49ba72b9dd service placement] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:17:47.942852 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8f6eed-4e24-42bd-9c51-ae49ba72b9dd service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:17:47.944658 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-ad8f6eed-4e24-42bd-9c51-ae49ba72b9dd service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:17:47.957306 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8f6eed-4e24-42bd-9c51-ae49ba72b9dd service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:17:47.958905 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8f6eed-4e24-42bd-9c51-ae49ba72b9dd service placement] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:17:47.969309 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1129/2253] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:17:47 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:17:57.964889 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=32 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:17:58.833569 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d344cd-460b-4b06-99a3-92d6fa9b3650 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:17:58.834049 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d344cd-460b-4b06-99a3-92d6fa9b3650 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:17:58.834142 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d344cd-460b-4b06-99a3-92d6fa9b3650 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:17:58.858278 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-d5d344cd-460b-4b06-99a3-92d6fa9b3650 None None] MFA Rules not processed for user `93207b858d294d049fa56d45810a40aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:17:58.859426 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-d5d344cd-460b-4b06-99a3-92d6fa9b3650 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:17:58.868364 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1125/2254] 10.222.0.42 () {58 vars in 967 bytes} [Mon Jul 25 12:17:58 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 12:17:58.882641 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8f6eed-4e24-42bd-9c51-ae49ba72b9dd service placement] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:17:58.894480 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8f6eed-4e24-42bd-9c51-ae49ba72b9dd service placement] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:17:58.899537 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effeb579-e08c-48ea-b46d-e6b9be92600b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bd782ea66364eeb96592096eae63d4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:17:58.900212 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effeb579-e08c-48ea-b46d-e6b9be92600b service swift] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:17:58.900502 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effeb579-e08c-48ea-b46d-e6b9be92600b service swift] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:17:58.900785 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effeb579-e08c-48ea-b46d-e6b9be92600b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:17:58.910337 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-effeb579-e08c-48ea-b46d-e6b9be92600b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:17:58.911967 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-effeb579-e08c-48ea-b46d-e6b9be92600b service swift] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:17:58.920731 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1130/2255] 10.222.0.42 () {60 vars in 1384 bytes} [Mon Jul 25 12:17:58 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 25 12:18:08.848473 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=3 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:18:08.917026 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=6 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:18:38.584070 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5697689-0907-4674-88e4-7e287088734c None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:18:38.584604 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5697689-0907-4674-88e4-7e287088734c None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:18:38.584746 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5697689-0907-4674-88e4-7e287088734c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:18:38.603401 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-d5697689-0907-4674-88e4-7e287088734c None None] MFA Rules not processed for user `3848eae5b8df48eb9816ba0d7485f7c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:18:38.604662 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-d5697689-0907-4674-88e4-7e287088734c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:18:38.667339 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1126/2256] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:18:38 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:18:48.660765 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=25 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:18:51.114478 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce815ba-f425-44d9-b60e-deecf5ea98f0 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:18:51.114989 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce815ba-f425-44d9-b60e-deecf5ea98f0 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:18:51.115123 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce815ba-f425-44d9-b60e-deecf5ea98f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:18:51.136477 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-5ce815ba-f425-44d9-b60e-deecf5ea98f0 None None] MFA Rules not processed for user `93207b858d294d049fa56d45810a40aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:18:51.137824 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-5ce815ba-f425-44d9-b60e-deecf5ea98f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:18:51.145280 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1131/2257] 10.222.0.42 () {58 vars in 967 bytes} [Mon Jul 25 12:18:51 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:18:51.171613 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5697689-0907-4674-88e4-7e287088734c None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:18:51.184564 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5697689-0907-4674-88e4-7e287088734c None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:18:51.191781 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c90ef1-94ad-44df-825c-444c7599842b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bd782ea66364eeb96592096eae63d4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:18:51.192804 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c90ef1-94ad-44df-825c-444c7599842b service swift] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:18:51.193294 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c90ef1-94ad-44df-825c-444c7599842b service swift] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:18:51.193785 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c90ef1-94ad-44df-825c-444c7599842b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:18:51.204393 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c90ef1-94ad-44df-825c-444c7599842b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:18:51.206260 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c90ef1-94ad-44df-825c-444c7599842b service swift] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:18:51.223693 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1127/2258] 10.222.0.42 () {60 vars in 1384 bytes} [Mon Jul 25 12:18:51 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 25 12:18:53.469529 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af804807-5736-419e-ad1b-36c46b2c321a None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:18:53.469963 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af804807-5736-419e-ad1b-36c46b2c321a None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:18:53.470140 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af804807-5736-419e-ad1b-36c46b2c321a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:18:53.504302 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-af804807-5736-419e-ad1b-36c46b2c321a None None] MFA Rules not processed for user `d2d6f10a6fe841e5b1d12bce78f97d91`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:18:53.505559 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-af804807-5736-419e-ad1b-36c46b2c321a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:18:53.571407 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1132/2259] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:18:53 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:18:57.865536 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640e1949-5579-441d-83d2-efe488c91c1a None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:18:57.866057 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640e1949-5579-441d-83d2-efe488c91c1a None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:18:57.866141 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640e1949-5579-441d-83d2-efe488c91c1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:18:57.886044 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-640e1949-5579-441d-83d2-efe488c91c1a None None] MFA Rules not processed for user `a766d17b116f498fab07efb69c94e49e`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:18:57.887313 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-640e1949-5579-441d-83d2-efe488c91c1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:18:57.945820 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1128/2260] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:18:57 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:19:01.844620 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af804807-5736-419e-ad1b-36c46b2c321a None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:19:01.846129 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-af804807-5736-419e-ad1b-36c46b2c321a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:19:01.915670 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af804807-5736-419e-ad1b-36c46b2c321a None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:19:01.920383 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1962427a-dc5f-4b6a-9ffb-7180efd9d284 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:19:01.921263 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1962427a-dc5f-4b6a-9ffb-7180efd9d284 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:19:01.921558 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1962427a-dc5f-4b6a-9ffb-7180efd9d284 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:19:01.921843 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1962427a-dc5f-4b6a-9ffb-7180efd9d284 admin admin] PATH_INFO: `/v3/users/5ea6250801fc4f74b2180533687e8cf2` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:19:01.923057 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1962427a-dc5f-4b6a-9ffb-7180efd9d284 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5ea6250801fc4f74b2180533687e8cf2)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:19:01.925426 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1962427a-dc5f-4b6a-9ffb-7180efd9d284 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:19:01.972694 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-1962427a-dc5f-4b6a-9ffb-7180efd9d284 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5ea6250801fc4f74b2180533687e8cf2'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:19:01.973376 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-1962427a-dc5f-4b6a-9ffb-7180efd9d284 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5ea6250801fc4f74b2180533687e8cf2'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:19:01.980761 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-1962427a-dc5f-4b6a-9ffb-7180efd9d284 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5ea6250801fc4f74b2180533687e8cf2'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:19:01.992648 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-1962427a-dc5f-4b6a-9ffb-7180efd9d284 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5ea6250801fc4f74b2180533687e8cf2'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:19:01.999948 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-1962427a-dc5f-4b6a-9ffb-7180efd9d284 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5ea6250801fc4f74b2180533687e8cf2'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:19:02.006079 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1133/2261] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Jul 25 12:19:01 2022] DELETE /identity/v3/users/5ea6250801fc4f74b2180533687e8cf2 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:19:02.261767 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640e1949-5579-441d-83d2-efe488c91c1a None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:19:02.273575 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640e1949-5579-441d-83d2-efe488c91c1a None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:19:02.344581 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e21321-4802-4ce3-826c-c7ec24e1e170 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:19:02.345533 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e21321-4802-4ce3-826c-c7ec24e1e170 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:19:02.345886 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e21321-4802-4ce3-826c-c7ec24e1e170 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:19:02.346240 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e21321-4802-4ce3-826c-c7ec24e1e170 admin admin] PATH_INFO: `/v3/projects/f63781e2ea3b43b48fc5a065fea7bad4` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:19:02.347585 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e21321-4802-4ce3-826c-c7ec24e1e170 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f63781e2ea3b43b48fc5a065fea7bad4)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:19:02.349651 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e21321-4802-4ce3-826c-c7ec24e1e170 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:19:02.396487 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-b3e21321-4802-4ce3-826c-c7ec24e1e170 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f63781e2ea3b43b48fc5a065fea7bad4', 'request_id': 'req-b3e21321-4802-4ce3-826c-c7ec24e1e170'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:19:02.404034 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-b3e21321-4802-4ce3-826c-c7ec24e1e170 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f63781e2ea3b43b48fc5a065fea7bad4', 'request_id': 'req-b3e21321-4802-4ce3-826c-c7ec24e1e170'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:19:02.410284 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-b3e21321-4802-4ce3-826c-c7ec24e1e170 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f63781e2ea3b43b48fc5a065fea7bad4', 'request_id': 'req-b3e21321-4802-4ce3-826c-c7ec24e1e170'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:19:02.422326 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-b3e21321-4802-4ce3-826c-c7ec24e1e170 admin admin] The token cache is being invalidate because project f63781e2ea3b43b48fc5a065fea7bad4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 12:19:02.422624 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-b3e21321-4802-4ce3-826c-c7ec24e1e170 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:19:02.423278 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-b3e21321-4802-4ce3-826c-c7ec24e1e170 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:19:02.424914 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1129/2262] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Jul 25 12:19:02 2022] DELETE /identity/v3/projects/f63781e2ea3b43b48fc5a065fea7bad4 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:19:02.429572 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1962427a-dc5f-4b6a-9ffb-7180efd9d284 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:19:02.431108 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-1962427a-dc5f-4b6a-9ffb-7180efd9d284 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:19:02.445196 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1962427a-dc5f-4b6a-9ffb-7180efd9d284 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:19:02.458093 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:19:02.506745 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47a69b7-64c3-4026-9161-c0d3f3a7cec7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:19:02.507661 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47a69b7-64c3-4026-9161-c0d3f3a7cec7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:19:02.507956 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47a69b7-64c3-4026-9161-c0d3f3a7cec7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:19:02.508240 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47a69b7-64c3-4026-9161-c0d3f3a7cec7 admin admin] PATH_INFO: `/v3/users/457790bb4d2c4699801c4d51c660d9fa` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:19:02.509462 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47a69b7-64c3-4026-9161-c0d3f3a7cec7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=457790bb4d2c4699801c4d51c660d9fa)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:19:02.511378 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47a69b7-64c3-4026-9161-c0d3f3a7cec7 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:19:02.562070 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-b47a69b7-64c3-4026-9161-c0d3f3a7cec7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '457790bb4d2c4699801c4d51c660d9fa'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:19:02.562783 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-b47a69b7-64c3-4026-9161-c0d3f3a7cec7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '457790bb4d2c4699801c4d51c660d9fa'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:19:02.570974 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-b47a69b7-64c3-4026-9161-c0d3f3a7cec7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '457790bb4d2c4699801c4d51c660d9fa'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:19:02.583890 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-b47a69b7-64c3-4026-9161-c0d3f3a7cec7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '457790bb4d2c4699801c4d51c660d9fa'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:19:02.590968 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-b47a69b7-64c3-4026-9161-c0d3f3a7cec7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '457790bb4d2c4699801c4d51c660d9fa'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:19:02.597011 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1134/2263] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Jul 25 12:19:02 2022] DELETE /identity/v3/users/457790bb4d2c4699801c4d51c660d9fa => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:19:02.787678 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e21321-4802-4ce3-826c-c7ec24e1e170 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:19:02.789181 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-b3e21321-4802-4ce3-826c-c7ec24e1e170 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:19:02.805509 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e21321-4802-4ce3-826c-c7ec24e1e170 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:19:02.825252 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:19:02.865137 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e35dd02-054c-4ec5-a2df-61d39061d160 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:19:02.865993 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e35dd02-054c-4ec5-a2df-61d39061d160 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:19:02.866311 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e35dd02-054c-4ec5-a2df-61d39061d160 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:19:02.866588 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e35dd02-054c-4ec5-a2df-61d39061d160 admin admin] PATH_INFO: `/v3/projects/ff7fb61a835d438692cf3f64a1c552e1` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:19:02.867840 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e35dd02-054c-4ec5-a2df-61d39061d160 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ff7fb61a835d438692cf3f64a1c552e1)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:19:02.869750 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e35dd02-054c-4ec5-a2df-61d39061d160 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:19:02.915787 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-7e35dd02-054c-4ec5-a2df-61d39061d160 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ff7fb61a835d438692cf3f64a1c552e1', 'request_id': 'req-7e35dd02-054c-4ec5-a2df-61d39061d160'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:19:02.924121 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-7e35dd02-054c-4ec5-a2df-61d39061d160 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ff7fb61a835d438692cf3f64a1c552e1', 'request_id': 'req-7e35dd02-054c-4ec5-a2df-61d39061d160'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:19:02.929675 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-7e35dd02-054c-4ec5-a2df-61d39061d160 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ff7fb61a835d438692cf3f64a1c552e1', 'request_id': 'req-7e35dd02-054c-4ec5-a2df-61d39061d160'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:19:02.940362 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-7e35dd02-054c-4ec5-a2df-61d39061d160 admin admin] The token cache is being invalidate because project ff7fb61a835d438692cf3f64a1c552e1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 12:19:02.940658 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-7e35dd02-054c-4ec5-a2df-61d39061d160 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:19:02.941379 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-7e35dd02-054c-4ec5-a2df-61d39061d160 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:19:02.942950 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1130/2264] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Jul 25 12:19:02 2022] DELETE /identity/v3/projects/ff7fb61a835d438692cf3f64a1c552e1 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:19:12.593593 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=45,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:19:12.937609 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=41,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:19:19.690028 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce78cd1c-4110-4f83-8848-f8aa78457f85 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:19:19.690560 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce78cd1c-4110-4f83-8848-f8aa78457f85 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:19:19.690664 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce78cd1c-4110-4f83-8848-f8aa78457f85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:19:19.710257 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-ce78cd1c-4110-4f83-8848-f8aa78457f85 None None] MFA Rules not processed for user `c5b98bbcc2d640de837d141f83dfe924`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:19:19.711460 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-ce78cd1c-4110-4f83-8848-f8aa78457f85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:19:19.765495 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1135/2265] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:19:19 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:19:29.758239 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=25 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:19:42.108302 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e35dd02-054c-4ec5-a2df-61d39061d160 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:19:42.109745 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-7e35dd02-054c-4ec5-a2df-61d39061d160 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:19:42.165907 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e35dd02-054c-4ec5-a2df-61d39061d160 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:19:42.169519 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88d16dd-a74b-4256-95aa-79a7d5b1d8b5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f89ce6f0cf04373a414d927fa0f8a7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:19:42.170152 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88d16dd-a74b-4256-95aa-79a7d5b1d8b5 service cinder] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:19:42.170436 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88d16dd-a74b-4256-95aa-79a7d5b1d8b5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:19:42.170715 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88d16dd-a74b-4256-95aa-79a7d5b1d8b5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:19:42.172471 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-e88d16dd-a74b-4256-95aa-79a7d5b1d8b5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:19:42.241046 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88d16dd-a74b-4256-95aa-79a7d5b1d8b5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:19:42.242482 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88d16dd-a74b-4256-95aa-79a7d5b1d8b5 service cinder] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:19:42.253351 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1131/2266] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:19:42 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:19:42.978304 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce78cd1c-4110-4f83-8848-f8aa78457f85 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:19:42.980308 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-ce78cd1c-4110-4f83-8848-f8aa78457f85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:19:42.992368 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce78cd1c-4110-4f83-8848-f8aa78457f85 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:19:42.996643 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1ffd05-9deb-4e01-99ec-644c705ecccb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f89ce6f0cf04373a414d927fa0f8a7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:19:42.997292 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1ffd05-9deb-4e01-99ec-644c705ecccb service cinder] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:19:42.997579 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1ffd05-9deb-4e01-99ec-644c705ecccb service cinder] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:19:42.997860 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1ffd05-9deb-4e01-99ec-644c705ecccb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:19:42.999653 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-ef1ffd05-9deb-4e01-99ec-644c705ecccb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:19:43.012939 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1ffd05-9deb-4e01-99ec-644c705ecccb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:19:43.014353 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1ffd05-9deb-4e01-99ec-644c705ecccb service cinder] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:19:43.025485 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1136/2267] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:19:42 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:19:52.248563 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=54 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:19:53.020634 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=6 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:19:54.104671 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88d16dd-a74b-4256-95aa-79a7d5b1d8b5 service cinder] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:19:54.113383 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88d16dd-a74b-4256-95aa-79a7d5b1d8b5 service cinder] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:19:54.117337 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadf7210-3185-4722-b97f-1b18766c3939 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f89ce6f0cf04373a414d927fa0f8a7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:19:54.117847 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadf7210-3185-4722-b97f-1b18766c3939 service cinder] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:19:54.118198 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadf7210-3185-4722-b97f-1b18766c3939 service cinder] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:19:54.118569 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadf7210-3185-4722-b97f-1b18766c3939 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:19:54.120676 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-dadf7210-3185-4722-b97f-1b18766c3939 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:19:54.183781 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dadf7210-3185-4722-b97f-1b18766c3939 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:19:54.185245 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dadf7210-3185-4722-b97f-1b18766c3939 service cinder] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:19:54.195094 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1132/2268] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:19:54 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:19:55.348992 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3cf311-c110-49f7-a5d3-966c51605036 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:19:55.349505 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3cf311-c110-49f7-a5d3-966c51605036 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:19:55.349613 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3cf311-c110-49f7-a5d3-966c51605036 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:19:55.369265 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-7f3cf311-c110-49f7-a5d3-966c51605036 None None] MFA Rules not processed for user `3848eae5b8df48eb9816ba0d7485f7c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:19:55.370492 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-7f3cf311-c110-49f7-a5d3-966c51605036 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:19:55.424183 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1137/2269] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:19:55 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:19:56.029664 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadf7210-3185-4722-b97f-1b18766c3939 service cinder] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:19:56.031182 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-dadf7210-3185-4722-b97f-1b18766c3939 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:19:56.109705 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadf7210-3185-4722-b97f-1b18766c3939 service cinder] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:19:56.114678 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b1366c-cecf-4340-96a6-ba6d8c1ac31b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8012afab6cac4358ab878732e58d0689', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:19:56.115326 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b1366c-cecf-4340-96a6-ba6d8c1ac31b service placement] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:19:56.115649 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b1366c-cecf-4340-96a6-ba6d8c1ac31b service placement] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:19:56.115933 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b1366c-cecf-4340-96a6-ba6d8c1ac31b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:19:56.117694 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-98b1366c-cecf-4340-96a6-ba6d8c1ac31b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:19:56.129484 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b1366c-cecf-4340-96a6-ba6d8c1ac31b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:19:56.131095 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b1366c-cecf-4340-96a6-ba6d8c1ac31b service placement] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:19:56.141602 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1133/2270] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:19:56 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:19:56.786852 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3cf311-c110-49f7-a5d3-966c51605036 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:19:56.788326 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-7f3cf311-c110-49f7-a5d3-966c51605036 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:19:56.806909 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3cf311-c110-49f7-a5d3-966c51605036 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:19:56.812914 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa320751-2044-43f1-bd1f-2eb90339db16 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f67142a1193d47c7bd33d7040268233c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:19:56.813681 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa320751-2044-43f1-bd1f-2eb90339db16 service nova] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:19:56.814069 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa320751-2044-43f1-bd1f-2eb90339db16 service nova] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:19:56.814481 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa320751-2044-43f1-bd1f-2eb90339db16 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:19:56.816511 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-fa320751-2044-43f1-bd1f-2eb90339db16 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:19:56.888474 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa320751-2044-43f1-bd1f-2eb90339db16 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:19:56.890095 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa320751-2044-43f1-bd1f-2eb90339db16 service nova] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:19:56.900699 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1138/2271] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:19:56 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:19:57.091948 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b1366c-cecf-4340-96a6-ba6d8c1ac31b service placement] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:19:57.105914 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b1366c-cecf-4340-96a6-ba6d8c1ac31b service placement] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:19:57.111336 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7067e66d-d8ef-408f-9ee4-aea6c748cc35 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f67142a1193d47c7bd33d7040268233c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:19:57.112095 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7067e66d-d8ef-408f-9ee4-aea6c748cc35 service nova] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:19:57.112381 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7067e66d-d8ef-408f-9ee4-aea6c748cc35 service nova] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:19:57.112662 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7067e66d-d8ef-408f-9ee4-aea6c748cc35 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:19:57.114444 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-7067e66d-d8ef-408f-9ee4-aea6c748cc35 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:19:57.126758 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7067e66d-d8ef-408f-9ee4-aea6c748cc35 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:19:57.128477 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7067e66d-d8ef-408f-9ee4-aea6c748cc35 service nova] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:19:57.139067 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1134/2272] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:19:57 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:20:05.120335 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa320751-2044-43f1-bd1f-2eb90339db16 service nova] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:20:05.121717 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-fa320751-2044-43f1-bd1f-2eb90339db16 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:20:05.177467 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa320751-2044-43f1-bd1f-2eb90339db16 service nova] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:20:05.181959 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c81033b-c964-4421-a9e5-dad6c4952a24 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51733781201243e8abf9eca39b207750', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:20:05.182831 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c81033b-c964-4421-a9e5-dad6c4952a24 service neutron] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:20:05.183181 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c81033b-c964-4421-a9e5-dad6c4952a24 service neutron] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:20:05.183624 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c81033b-c964-4421-a9e5-dad6c4952a24 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:20:05.186342 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-4c81033b-c964-4421-a9e5-dad6c4952a24 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:20:05.247146 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:20:05.251734 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c81033b-c964-4421-a9e5-dad6c4952a24 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:20:05.253205 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c81033b-c964-4421-a9e5-dad6c4952a24 service neutron] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:20:05.263884 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1139/2273] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:20:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:20:07.134022 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=66 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:20:08.565720 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7067e66d-d8ef-408f-9ee4-aea6c748cc35 service nova] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:20:08.567208 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-7067e66d-d8ef-408f-9ee4-aea6c748cc35 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:20:08.586161 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7067e66d-d8ef-408f-9ee4-aea6c748cc35 service nova] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:20:08.590964 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3445bb-86c7-46af-a615-6c96e0cb4482 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:20:08.591863 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3445bb-86c7-46af-a615-6c96e0cb4482 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:20:08.592149 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3445bb-86c7-46af-a615-6c96e0cb4482 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:20:08.592424 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3445bb-86c7-46af-a615-6c96e0cb4482 admin admin] PATH_INFO: `/v3/users/a766d17b116f498fab07efb69c94e49e` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:20:08.593625 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3445bb-86c7-46af-a615-6c96e0cb4482 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a766d17b116f498fab07efb69c94e49e)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:20:08.595692 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3445bb-86c7-46af-a615-6c96e0cb4482 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:20:08.644350 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-9c3445bb-86c7-46af-a615-6c96e0cb4482 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a766d17b116f498fab07efb69c94e49e'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:20:08.651307 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-9c3445bb-86c7-46af-a615-6c96e0cb4482 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a766d17b116f498fab07efb69c94e49e'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:20:08.658805 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-9c3445bb-86c7-46af-a615-6c96e0cb4482 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a766d17b116f498fab07efb69c94e49e'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:20:08.675202 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-9c3445bb-86c7-46af-a615-6c96e0cb4482 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a766d17b116f498fab07efb69c94e49e'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:20:08.675935 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-9c3445bb-86c7-46af-a615-6c96e0cb4482 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a766d17b116f498fab07efb69c94e49e'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:20:08.683299 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1135/2274] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Jul 25 12:20:08 2022] DELETE /identity/v3/users/a766d17b116f498fab07efb69c94e49e => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:20:08.861215 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c81033b-c964-4421-a9e5-dad6c4952a24 service neutron] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:20:08.871466 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c81033b-c964-4421-a9e5-dad6c4952a24 service neutron] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:20:08.930617 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedfccd2-a97c-4270-9333-80682d9b4466 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:20:08.931484 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedfccd2-a97c-4270-9333-80682d9b4466 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:20:08.931802 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedfccd2-a97c-4270-9333-80682d9b4466 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:20:08.932082 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedfccd2-a97c-4270-9333-80682d9b4466 admin admin] PATH_INFO: `/v3/projects/2d9f339447114912b0c92b99e028fbbf` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:20:08.933313 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fedfccd2-a97c-4270-9333-80682d9b4466 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2d9f339447114912b0c92b99e028fbbf)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:20:08.935229 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fedfccd2-a97c-4270-9333-80682d9b4466 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:20:08.991043 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-fedfccd2-a97c-4270-9333-80682d9b4466 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2d9f339447114912b0c92b99e028fbbf', 'request_id': 'req-fedfccd2-a97c-4270-9333-80682d9b4466'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:20:09.001617 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-fedfccd2-a97c-4270-9333-80682d9b4466 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2d9f339447114912b0c92b99e028fbbf', 'request_id': 'req-fedfccd2-a97c-4270-9333-80682d9b4466'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:20:09.014476 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-fedfccd2-a97c-4270-9333-80682d9b4466 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2d9f339447114912b0c92b99e028fbbf', 'request_id': 'req-fedfccd2-a97c-4270-9333-80682d9b4466'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:20:09.026916 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-fedfccd2-a97c-4270-9333-80682d9b4466 admin admin] The token cache is being invalidate because project 2d9f339447114912b0c92b99e028fbbf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80298) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 12:20:09.027535 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-fedfccd2-a97c-4270-9333-80682d9b4466 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:20:09.028263 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-fedfccd2-a97c-4270-9333-80682d9b4466 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:20:09.030374 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1140/2275] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Jul 25 12:20:08 2022] DELETE /identity/v3/projects/2d9f339447114912b0c92b99e028fbbf => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:20:09.035101 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f041a5-e00f-45c6-875b-83bc0093c65f None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:20:09.035395 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f041a5-e00f-45c6-875b-83bc0093c65f None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:20:09.035718 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f041a5-e00f-45c6-875b-83bc0093c65f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:20:09.059740 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-c7f041a5-e00f-45c6-875b-83bc0093c65f None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:20:09.060964 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-c7f041a5-e00f-45c6-875b-83bc0093c65f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:20:09.128860 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1136/2276] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:20:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:20:09.136428 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332457d2-8c54-4b70-b772-988408079a58 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:20:09.136797 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332457d2-8c54-4b70-b772-988408079a58 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:20:09.137305 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332457d2-8c54-4b70-b772-988408079a58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:20:09.163998 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-332457d2-8c54-4b70-b772-988408079a58 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:20:09.165829 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-332457d2-8c54-4b70-b772-988408079a58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:20:09.175738 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1141/2277] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:20:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:20:09.179966 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f041a5-e00f-45c6-875b-83bc0093c65f None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:20:09.190037 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f041a5-e00f-45c6-875b-83bc0093c65f None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:20:09.195403 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130b1cb5-9ae0-4cb6-be10-17ad067bb7d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:20:09.196738 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130b1cb5-9ae0-4cb6-be10-17ad067bb7d6 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:20:09.197096 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130b1cb5-9ae0-4cb6-be10-17ad067bb7d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:20:09.197447 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130b1cb5-9ae0-4cb6-be10-17ad067bb7d6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:20:09.198257 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130b1cb5-9ae0-4cb6-be10-17ad067bb7d6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:20:09.198637 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130b1cb5-9ae0-4cb6-be10-17ad067bb7d6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:20:09.200420 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130b1cb5-9ae0-4cb6-be10-17ad067bb7d6 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:20:09.213934 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1137/2278] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Jul 25 12:20:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:20:09.218370 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332457d2-8c54-4b70-b772-988408079a58 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:20:09.230978 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332457d2-8c54-4b70-b772-988408079a58 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:20:09.236205 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2883a3-04d3-4cba-b604-e7665aa7441e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:20:09.236781 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2883a3-04d3-4cba-b604-e7665aa7441e admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:20:09.237089 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2883a3-04d3-4cba-b604-e7665aa7441e admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:20:09.237358 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2883a3-04d3-4cba-b604-e7665aa7441e admin admin] PATH_INFO: `/v3/projects` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:20:09.238081 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2883a3-04d3-4cba-b604-e7665aa7441e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:20:09.240095 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2883a3-04d3-4cba-b604-e7665aa7441e admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:20:09.271964 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1142/2279] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Jul 25 12:20:09 2022] POST /identity/v3/projects => generated 405 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:20:09.276754 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130b1cb5-9ae0-4cb6-be10-17ad067bb7d6 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:20:09.287244 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130b1cb5-9ae0-4cb6-be10-17ad067bb7d6 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:20:09.347966 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d85cd96-1352-4048-a7d4-16badd14146c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:20:09.348646 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d85cd96-1352-4048-a7d4-16badd14146c admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:20:09.348936 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d85cd96-1352-4048-a7d4-16badd14146c admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:20:09.349220 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d85cd96-1352-4048-a7d4-16badd14146c admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:20:09.349913 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d85cd96-1352-4048-a7d4-16badd14146c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:20:09.351903 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d85cd96-1352-4048-a7d4-16badd14146c admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:20:09.359897 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:20:09.371056 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2883a3-04d3-4cba-b604-e7665aa7441e admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:20:09.372949 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-8a2883a3-04d3-4cba-b604-e7665aa7441e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:20:09.375752 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1138/2280] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Jul 25 12:20:09 2022] POST /identity/v3/users => generated 303 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:20:09.379873 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d85cd96-1352-4048-a7d4-16badd14146c admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:20:09.390219 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d85cd96-1352-4048-a7d4-16badd14146c admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:20:09.394873 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86881cb3-35a2-4032-8c7b-f0cf798e0c17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:20:09.395530 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86881cb3-35a2-4032-8c7b-f0cf798e0c17 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:20:09.395845 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86881cb3-35a2-4032-8c7b-f0cf798e0c17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:20:09.396131 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86881cb3-35a2-4032-8c7b-f0cf798e0c17 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:20:09.396860 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86881cb3-35a2-4032-8c7b-f0cf798e0c17 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:20:09.398486 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86881cb3-35a2-4032-8c7b-f0cf798e0c17 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:20:09.409921 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1139/2281] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Jul 25 12:20:09 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 12:20:09.414576 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86881cb3-35a2-4032-8c7b-f0cf798e0c17 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:20:09.425254 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86881cb3-35a2-4032-8c7b-f0cf798e0c17 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:20:09.429866 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a60525a-b67d-48d9-9172-4789d8ffff30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:20:09.430565 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a60525a-b67d-48d9-9172-4789d8ffff30 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:20:09.431486 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a60525a-b67d-48d9-9172-4789d8ffff30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:20:09.431879 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a60525a-b67d-48d9-9172-4789d8ffff30 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:20:09.432771 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a60525a-b67d-48d9-9172-4789d8ffff30 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:20:09.434377 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a60525a-b67d-48d9-9172-4789d8ffff30 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:20:09.435379 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2883a3-04d3-4cba-b604-e7665aa7441e admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:20:09.439461 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d726c596-a9b8-4bcc-adba-efc20b73b04b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51733781201243e8abf9eca39b207750', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:20:09.440151 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d726c596-a9b8-4bcc-adba-efc20b73b04b service neutron] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:20:09.440438 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d726c596-a9b8-4bcc-adba-efc20b73b04b service neutron] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:20:09.440776 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d726c596-a9b8-4bcc-adba-efc20b73b04b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:20:09.443189 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-d726c596-a9b8-4bcc-adba-efc20b73b04b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:20:09.448290 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1140/2282] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Jul 25 12:20:09 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 12:20:09.452444 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a60525a-b67d-48d9-9172-4789d8ffff30 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:20:09.454407 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d726c596-a9b8-4bcc-adba-efc20b73b04b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:20:09.455933 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d726c596-a9b8-4bcc-adba-efc20b73b04b service neutron] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:20:09.460087 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:20:09.463883 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a60525a-b67d-48d9-9172-4789d8ffff30 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:20:09.467500 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1143/2283] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:20:09 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 12:20:09.469034 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae44b8b-388c-4713-b896-817a49748a90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:20:09.469912 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae44b8b-388c-4713-b896-817a49748a90 admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:20:09.470197 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae44b8b-388c-4713-b896-817a49748a90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:20:09.470478 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae44b8b-388c-4713-b896-817a49748a90 admin admin] PATH_INFO: `/v3/projects/15451f1ce9b8425a80be16b2bd72c062/users/edd1ffe55fc64cffb6e4a475d3b3a9e2/roles/472313ae69174578824df0bed6390744` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:20:09.486988 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae44b8b-388c-4713-b896-817a49748a90 admin admin] RBAC: Authorizing `identity:create_grant(project_id=15451f1ce9b8425a80be16b2bd72c062, user_id=edd1ffe55fc64cffb6e4a475d3b3a9e2, role_id=472313ae69174578824df0bed6390744)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:20:09.489849 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae44b8b-388c-4713-b896-817a49748a90 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:20:09.503361 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1141/2284] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Jul 25 12:20:09 2022] PUT /identity/v3/projects/15451f1ce9b8425a80be16b2bd72c062/users/edd1ffe55fc64cffb6e4a475d3b3a9e2/roles/472313ae69174578824df0bed6390744 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:20:09.515467 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab672844-c1cf-493c-944d-89e8cf0bd36f None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:20:09.515800 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab672844-c1cf-493c-944d-89e8cf0bd36f None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:20:09.516088 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab672844-c1cf-493c-944d-89e8cf0bd36f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:20:09.544636 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-ab672844-c1cf-493c-944d-89e8cf0bd36f None None] MFA Rules not processed for user `edd1ffe55fc64cffb6e4a475d3b3a9e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:20:09.545870 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-ab672844-c1cf-493c-944d-89e8cf0bd36f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:20:09.611284 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1144/2285] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:20:09 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:20:09.615861 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae44b8b-388c-4713-b896-817a49748a90 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:20:09.678186 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae44b8b-388c-4713-b896-817a49748a90 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:20:09.683474 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe9dd53-8e07-4301-ba19-31156e5cd887 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:20:09.684161 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe9dd53-8e07-4301-ba19-31156e5cd887 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:20:09.684448 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe9dd53-8e07-4301-ba19-31156e5cd887 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:20:09.684728 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe9dd53-8e07-4301-ba19-31156e5cd887 admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:20:09.685431 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe9dd53-8e07-4301-ba19-31156e5cd887 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:20:09.687374 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe9dd53-8e07-4301-ba19-31156e5cd887 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:20:09.713504 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1142/2286] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Jul 25 12:20:09 2022] POST /identity/v3/projects => generated 405 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 12:20:09.717760 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab672844-c1cf-493c-944d-89e8cf0bd36f None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:20:09.729334 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab672844-c1cf-493c-944d-89e8cf0bd36f None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:20:09.793852 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87912693-8b87-402f-a832-d92ea624856c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:20:09.794530 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87912693-8b87-402f-a832-d92ea624856c admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:20:09.794820 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87912693-8b87-402f-a832-d92ea624856c admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:20:09.795100 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87912693-8b87-402f-a832-d92ea624856c admin admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:20:09.795838 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87912693-8b87-402f-a832-d92ea624856c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:20:09.797763 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87912693-8b87-402f-a832-d92ea624856c admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:20:09.817777 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1145/2287] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Jul 25 12:20:09 2022] POST /identity/v3/users => generated 309 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:20:09.822414 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe9dd53-8e07-4301-ba19-31156e5cd887 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:20:09.832847 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe9dd53-8e07-4301-ba19-31156e5cd887 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:20:09.837519 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c75152-e4af-4c50-97d0-89437a063e91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:20:09.838179 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c75152-e4af-4c50-97d0-89437a063e91 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:20:09.838469 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c75152-e4af-4c50-97d0-89437a063e91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:20:09.838753 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c75152-e4af-4c50-97d0-89437a063e91 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:20:09.839509 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c75152-e4af-4c50-97d0-89437a063e91 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:20:09.841133 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c75152-e4af-4c50-97d0-89437a063e91 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:20:09.851531 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1143/2288] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Jul 25 12:20:09 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:20:09.856124 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87912693-8b87-402f-a832-d92ea624856c admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:20:09.865997 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87912693-8b87-402f-a832-d92ea624856c admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:20:09.871259 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef07388-9123-4029-bee3-6d2a6e379149 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:20:09.872169 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef07388-9123-4029-bee3-6d2a6e379149 admin admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:20:09.872458 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef07388-9123-4029-bee3-6d2a6e379149 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:20:09.872740 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef07388-9123-4029-bee3-6d2a6e379149 admin admin] PATH_INFO: `/v3/projects/b4655e4ca08e46b9bf3338a041c753c9/users/2a0472e0bec945fe9877fb67660e1852/roles/a7d20885b6d94928a34b3cc5b3a00006` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:20:09.887240 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef07388-9123-4029-bee3-6d2a6e379149 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b4655e4ca08e46b9bf3338a041c753c9, user_id=2a0472e0bec945fe9877fb67660e1852, role_id=a7d20885b6d94928a34b3cc5b3a00006)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:20:09.889931 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef07388-9123-4029-bee3-6d2a6e379149 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:20:09.902971 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1146/2289] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Jul 25 12:20:09 2022] PUT /identity/v3/projects/b4655e4ca08e46b9bf3338a041c753c9/users/2a0472e0bec945fe9877fb67660e1852/roles/a7d20885b6d94928a34b3cc5b3a00006 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:20:09.906851 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c75152-e4af-4c50-97d0-89437a063e91 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:20:09.917249 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c75152-e4af-4c50-97d0-89437a063e91 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:20:09.972317 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523d7282-d2b1-4c82-9bc3-0233849fd6da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:20:09.972986 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523d7282-d2b1-4c82-9bc3-0233849fd6da admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:20:09.973276 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523d7282-d2b1-4c82-9bc3-0233849fd6da admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:20:09.973558 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523d7282-d2b1-4c82-9bc3-0233849fd6da admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:20:09.974288 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523d7282-d2b1-4c82-9bc3-0233849fd6da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:20:09.975927 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523d7282-d2b1-4c82-9bc3-0233849fd6da admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:20:09.986396 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1144/2290] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Jul 25 12:20:09 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:20:09.990613 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef07388-9123-4029-bee3-6d2a6e379149 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:20:10.001272 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef07388-9123-4029-bee3-6d2a6e379149 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:20:10.005894 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae565ac-bd00-49ad-8212-f422646d2aad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:20:10.006769 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae565ac-bd00-49ad-8212-f422646d2aad admin admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:20:10.007071 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae565ac-bd00-49ad-8212-f422646d2aad admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:20:10.007350 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae565ac-bd00-49ad-8212-f422646d2aad admin admin] PATH_INFO: `/v3/projects/b4655e4ca08e46b9bf3338a041c753c9/users/2a0472e0bec945fe9877fb67660e1852/roles/a7d20885b6d94928a34b3cc5b3a00006` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:20:10.009339 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae565ac-bd00-49ad-8212-f422646d2aad admin admin] RBAC: Authorizing `identity:create_grant(project_id=b4655e4ca08e46b9bf3338a041c753c9, user_id=2a0472e0bec945fe9877fb67660e1852, role_id=a7d20885b6d94928a34b3cc5b3a00006)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:20:10.012028 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae565ac-bd00-49ad-8212-f422646d2aad admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:20:10.021922 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1147/2291] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Jul 25 12:20:09 2022] PUT /identity/v3/projects/b4655e4ca08e46b9bf3338a041c753c9/users/2a0472e0bec945fe9877fb67660e1852/roles/a7d20885b6d94928a34b3cc5b3a00006 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:20:10.032390 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc23873b-a79a-4d23-b781-f1e87278b1b8 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:20:10.032594 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc23873b-a79a-4d23-b781-f1e87278b1b8 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:20:10.032872 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc23873b-a79a-4d23-b781-f1e87278b1b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:20:10.060413 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:20:10.063489 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-bc23873b-a79a-4d23-b781-f1e87278b1b8 None None] MFA Rules not processed for user `2a0472e0bec945fe9877fb67660e1852`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:20:10.065243 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-bc23873b-a79a-4d23-b781-f1e87278b1b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:20:10.132543 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1145/2292] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:20:10 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:20:10.146579 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae565ac-bd00-49ad-8212-f422646d2aad admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:20:10.148354 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-fae565ac-bd00-49ad-8212-f422646d2aad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:20:10.194795 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:20:10.209687 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae565ac-bd00-49ad-8212-f422646d2aad admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:20:10.213446 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4a782e-1dee-4218-b92b-5f6901ad0842 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f89ce6f0cf04373a414d927fa0f8a7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:20:10.214134 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4a782e-1dee-4218-b92b-5f6901ad0842 service cinder] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:20:10.214407 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4a782e-1dee-4218-b92b-5f6901ad0842 service cinder] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:20:10.214693 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4a782e-1dee-4218-b92b-5f6901ad0842 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:20:10.224087 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4a782e-1dee-4218-b92b-5f6901ad0842 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:20:10.225517 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4a782e-1dee-4218-b92b-5f6901ad0842 service cinder] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:20:10.235697 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1148/2293] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:20:10 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:20:10.378888 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc23873b-a79a-4d23-b781-f1e87278b1b8 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:20:10.445687 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc23873b-a79a-4d23-b781-f1e87278b1b8 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:20:10.450671 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35c0ce1-c31b-4001-b681-74133b4716b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:20:10.451599 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35c0ce1-c31b-4001-b681-74133b4716b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:20:10.451870 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35c0ce1-c31b-4001-b681-74133b4716b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:20:10.452156 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35c0ce1-c31b-4001-b681-74133b4716b6 admin admin] PATH_INFO: `/v3/users/edd1ffe55fc64cffb6e4a475d3b3a9e2` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:20:10.453410 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35c0ce1-c31b-4001-b681-74133b4716b6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=edd1ffe55fc64cffb6e4a475d3b3a9e2)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:20:10.455525 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35c0ce1-c31b-4001-b681-74133b4716b6 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:20:10.506005 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-e35c0ce1-c31b-4001-b681-74133b4716b6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'edd1ffe55fc64cffb6e4a475d3b3a9e2'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:20:10.513386 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-e35c0ce1-c31b-4001-b681-74133b4716b6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'edd1ffe55fc64cffb6e4a475d3b3a9e2'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:20:10.520944 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-e35c0ce1-c31b-4001-b681-74133b4716b6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'edd1ffe55fc64cffb6e4a475d3b3a9e2'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:20:10.533569 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-e35c0ce1-c31b-4001-b681-74133b4716b6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'edd1ffe55fc64cffb6e4a475d3b3a9e2'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:20:10.534193 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-e35c0ce1-c31b-4001-b681-74133b4716b6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'edd1ffe55fc64cffb6e4a475d3b3a9e2'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:20:10.540682 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1146/2294] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Jul 25 12:20:10 2022] DELETE /identity/v3/users/edd1ffe55fc64cffb6e4a475d3b3a9e2 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:20:10.545226 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504742bd-13dd-4e4d-981a-b89ae2acce6c None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:20:10.545465 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504742bd-13dd-4e4d-981a-b89ae2acce6c None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:20:10.545754 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504742bd-13dd-4e4d-981a-b89ae2acce6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:20:10.564267 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-504742bd-13dd-4e4d-981a-b89ae2acce6c None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:20:10.565476 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-504742bd-13dd-4e4d-981a-b89ae2acce6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:20:10.625202 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1149/2295] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:20:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 12:20:10.636277 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35c0ce1-c31b-4001-b681-74133b4716b6 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:20:10.637676 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-e35c0ce1-c31b-4001-b681-74133b4716b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:20:10.659071 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:20:10.698402 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35c0ce1-c31b-4001-b681-74133b4716b6 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:20:10.702096 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959d4db5-cc88-44da-9317-d20ba64669d9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51733781201243e8abf9eca39b207750', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:20:10.702749 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959d4db5-cc88-44da-9317-d20ba64669d9 service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:20:10.703039 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959d4db5-cc88-44da-9317-d20ba64669d9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:20:10.703326 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959d4db5-cc88-44da-9317-d20ba64669d9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:20:10.713098 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959d4db5-cc88-44da-9317-d20ba64669d9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:20:10.714528 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959d4db5-cc88-44da-9317-d20ba64669d9 service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:20:10.723938 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1147/2296] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:20:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:20:10.994753 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504742bd-13dd-4e4d-981a-b89ae2acce6c None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:20:10.996334 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-504742bd-13dd-4e4d-981a-b89ae2acce6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:20:11.010097 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504742bd-13dd-4e4d-981a-b89ae2acce6c None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:20:11.014770 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b362deb5-20c5-4e85-950c-d802dd4ef6a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:20:11.015671 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b362deb5-20c5-4e85-950c-d802dd4ef6a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:20:11.015965 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b362deb5-20c5-4e85-950c-d802dd4ef6a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:20:11.016249 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b362deb5-20c5-4e85-950c-d802dd4ef6a3 admin admin] PATH_INFO: `/v3/projects/15451f1ce9b8425a80be16b2bd72c062` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:20:11.017445 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b362deb5-20c5-4e85-950c-d802dd4ef6a3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=15451f1ce9b8425a80be16b2bd72c062)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:20:11.019366 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b362deb5-20c5-4e85-950c-d802dd4ef6a3 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:20:11.066687 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-b362deb5-20c5-4e85-950c-d802dd4ef6a3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '15451f1ce9b8425a80be16b2bd72c062', 'request_id': 'req-b362deb5-20c5-4e85-950c-d802dd4ef6a3'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:20:11.074460 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-b362deb5-20c5-4e85-950c-d802dd4ef6a3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '15451f1ce9b8425a80be16b2bd72c062', 'request_id': 'req-b362deb5-20c5-4e85-950c-d802dd4ef6a3'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:20:11.080652 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-b362deb5-20c5-4e85-950c-d802dd4ef6a3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '15451f1ce9b8425a80be16b2bd72c062', 'request_id': 'req-b362deb5-20c5-4e85-950c-d802dd4ef6a3'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:20:11.093060 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-b362deb5-20c5-4e85-950c-d802dd4ef6a3 admin admin] The token cache is being invalidate because project 15451f1ce9b8425a80be16b2bd72c062 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80298) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 12:20:11.093354 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-b362deb5-20c5-4e85-950c-d802dd4ef6a3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:20:11.094013 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-b362deb5-20c5-4e85-950c-d802dd4ef6a3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:20:11.095639 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1150/2297] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Jul 25 12:20:10 2022] DELETE /identity/v3/projects/15451f1ce9b8425a80be16b2bd72c062 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:20:11.099821 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959d4db5-cc88-44da-9317-d20ba64669d9 service neutron] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:20:11.109939 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959d4db5-cc88-44da-9317-d20ba64669d9 service neutron] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:20:11.167708 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15454588-9800-427d-a735-c73534b93a4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:20:11.168582 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15454588-9800-427d-a735-c73534b93a4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:20:11.168870 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15454588-9800-427d-a735-c73534b93a4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:20:11.169154 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15454588-9800-427d-a735-c73534b93a4e admin admin] PATH_INFO: `/v3/users/2a0472e0bec945fe9877fb67660e1852` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:20:11.170348 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15454588-9800-427d-a735-c73534b93a4e admin admin] RBAC: Authorizing `identity:delete_user(user_id=2a0472e0bec945fe9877fb67660e1852)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:20:11.172307 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15454588-9800-427d-a735-c73534b93a4e admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:20:11.217685 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-15454588-9800-427d-a735-c73534b93a4e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2a0472e0bec945fe9877fb67660e1852'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:20:11.224461 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-15454588-9800-427d-a735-c73534b93a4e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2a0472e0bec945fe9877fb67660e1852'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:20:11.231315 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-15454588-9800-427d-a735-c73534b93a4e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2a0472e0bec945fe9877fb67660e1852'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:20:11.243766 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-15454588-9800-427d-a735-c73534b93a4e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2a0472e0bec945fe9877fb67660e1852'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:20:11.244396 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-15454588-9800-427d-a735-c73534b93a4e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2a0472e0bec945fe9877fb67660e1852'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:20:11.250167 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1148/2298] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Jul 25 12:20:11 2022] DELETE /identity/v3/users/2a0472e0bec945fe9877fb67660e1852 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:20:11.561822 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b362deb5-20c5-4e85-950c-d802dd4ef6a3 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:20:11.563258 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-b362deb5-20c5-4e85-950c-d802dd4ef6a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:20:11.577048 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b362deb5-20c5-4e85-950c-d802dd4ef6a3 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:20:11.642447 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ece3eb-14f4-45e4-b1ff-0edf91136dde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:20:11.643374 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ece3eb-14f4-45e4-b1ff-0edf91136dde admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:20:11.643713 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ece3eb-14f4-45e4-b1ff-0edf91136dde admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:20:11.643999 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ece3eb-14f4-45e4-b1ff-0edf91136dde admin admin] PATH_INFO: `/v3/projects/b4655e4ca08e46b9bf3338a041c753c9` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:20:11.645212 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ece3eb-14f4-45e4-b1ff-0edf91136dde admin admin] RBAC: Authorizing `identity:delete_project(project_id=b4655e4ca08e46b9bf3338a041c753c9)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:20:11.647134 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ece3eb-14f4-45e4-b1ff-0edf91136dde admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:20:11.655704 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:20:11.698900 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-d0ece3eb-14f4-45e4-b1ff-0edf91136dde admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b4655e4ca08e46b9bf3338a041c753c9', 'request_id': 'req-d0ece3eb-14f4-45e4-b1ff-0edf91136dde'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:20:11.706831 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-d0ece3eb-14f4-45e4-b1ff-0edf91136dde admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b4655e4ca08e46b9bf3338a041c753c9', 'request_id': 'req-d0ece3eb-14f4-45e4-b1ff-0edf91136dde'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:20:11.713338 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-d0ece3eb-14f4-45e4-b1ff-0edf91136dde admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b4655e4ca08e46b9bf3338a041c753c9', 'request_id': 'req-d0ece3eb-14f4-45e4-b1ff-0edf91136dde'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:20:11.725586 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-d0ece3eb-14f4-45e4-b1ff-0edf91136dde admin admin] The token cache is being invalidate because project b4655e4ca08e46b9bf3338a041c753c9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80298) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 12:20:11.725913 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-d0ece3eb-14f4-45e4-b1ff-0edf91136dde admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:20:11.726611 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-d0ece3eb-14f4-45e4-b1ff-0edf91136dde admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:20:11.728717 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1151/2299] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Jul 25 12:20:11 2022] DELETE /identity/v3/projects/b4655e4ca08e46b9bf3338a041c753c9 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:20:11.733635 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4b783b-24ef-4542-abef-c88517a4b814 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:20:11.734158 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4b783b-24ef-4542-abef-c88517a4b814 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:20:11.734372 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4b783b-24ef-4542-abef-c88517a4b814 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:20:11.754550 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-1e4b783b-24ef-4542-abef-c88517a4b814 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:20:11.755920 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-1e4b783b-24ef-4542-abef-c88517a4b814 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:20:11.800092 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:20:11.842030 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1149/2300] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:20:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 12:20:11.849817 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e9bb39-6e29-4e61-880f-a3e05458c94b None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:20:11.850117 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e9bb39-6e29-4e61-880f-a3e05458c94b None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:20:11.850408 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e9bb39-6e29-4e61-880f-a3e05458c94b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:20:11.871565 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-10e9bb39-6e29-4e61-880f-a3e05458c94b None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:20:11.872824 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-10e9bb39-6e29-4e61-880f-a3e05458c94b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:20:11.881053 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1152/2301] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:20:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:20:11.885290 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4b783b-24ef-4542-abef-c88517a4b814 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:20:11.895058 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4b783b-24ef-4542-abef-c88517a4b814 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:20:11.900242 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba21adb-459e-4023-9443-7b4fa46065f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:20:11.900898 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba21adb-459e-4023-9443-7b4fa46065f6 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:20:11.901183 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba21adb-459e-4023-9443-7b4fa46065f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:20:11.901464 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba21adb-459e-4023-9443-7b4fa46065f6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:20:11.902471 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba21adb-459e-4023-9443-7b4fa46065f6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:20:11.902780 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba21adb-459e-4023-9443-7b4fa46065f6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:20:11.904413 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba21adb-459e-4023-9443-7b4fa46065f6 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:20:11.916935 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1150/2302] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Jul 25 12:20:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 12:20:11.921265 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e9bb39-6e29-4e61-880f-a3e05458c94b None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:20:11.930978 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e9bb39-6e29-4e61-880f-a3e05458c94b None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:20:11.935678 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ee0b87-07a4-42e8-9319-10fa0c660546 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:20:11.936326 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ee0b87-07a4-42e8-9319-10fa0c660546 admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:20:11.936610 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ee0b87-07a4-42e8-9319-10fa0c660546 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:20:11.936886 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ee0b87-07a4-42e8-9319-10fa0c660546 admin admin] PATH_INFO: `/v3/projects` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:20:11.937618 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ee0b87-07a4-42e8-9319-10fa0c660546 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:20:11.939505 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ee0b87-07a4-42e8-9319-10fa0c660546 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:20:11.971867 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1153/2303] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Jul 25 12:20:11 2022] POST /identity/v3/projects => generated 377 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 12:20:11.972981 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba21adb-459e-4023-9443-7b4fa46065f6 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:20:11.983939 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba21adb-459e-4023-9443-7b4fa46065f6 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:20:12.041827 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7297ae9-b116-4503-a9f8-27906d1dc3bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:20:12.042504 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7297ae9-b116-4503-a9f8-27906d1dc3bc admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:20:12.042794 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7297ae9-b116-4503-a9f8-27906d1dc3bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:20:12.043076 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7297ae9-b116-4503-a9f8-27906d1dc3bc admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:20:12.043824 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7297ae9-b116-4503-a9f8-27906d1dc3bc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:20:12.045760 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7297ae9-b116-4503-a9f8-27906d1dc3bc admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:20:12.069789 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1151/2304] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Jul 25 12:20:11 2022] POST /identity/v3/users => generated 289 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 12:20:12.071135 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ee0b87-07a4-42e8-9319-10fa0c660546 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:20:12.082075 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ee0b87-07a4-42e8-9319-10fa0c660546 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:20:12.087659 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982ac17f-3bf1-49bb-8020-cc15afac8071 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:20:12.088321 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982ac17f-3bf1-49bb-8020-cc15afac8071 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:20:12.088611 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982ac17f-3bf1-49bb-8020-cc15afac8071 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:20:12.088894 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982ac17f-3bf1-49bb-8020-cc15afac8071 admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:20:12.089684 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982ac17f-3bf1-49bb-8020-cc15afac8071 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:20:12.091284 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982ac17f-3bf1-49bb-8020-cc15afac8071 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:20:12.103241 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1154/2305] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Jul 25 12:20:12 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:20:12.107185 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7297ae9-b116-4503-a9f8-27906d1dc3bc admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:20:12.118067 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7297ae9-b116-4503-a9f8-27906d1dc3bc admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:20:12.122749 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697c744b-47c5-4081-b5ab-1194b428b397 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:20:12.123410 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697c744b-47c5-4081-b5ab-1194b428b397 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:20:12.123732 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697c744b-47c5-4081-b5ab-1194b428b397 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:20:12.124010 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697c744b-47c5-4081-b5ab-1194b428b397 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:20:12.124749 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697c744b-47c5-4081-b5ab-1194b428b397 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:20:12.126349 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697c744b-47c5-4081-b5ab-1194b428b397 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:20:12.137548 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1152/2306] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Jul 25 12:20:12 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:20:12.141597 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982ac17f-3bf1-49bb-8020-cc15afac8071 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:20:12.152959 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982ac17f-3bf1-49bb-8020-cc15afac8071 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:20:12.158111 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7fd62a-cb46-488d-9414-6e122ccdef95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:20:12.159000 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7fd62a-cb46-488d-9414-6e122ccdef95 admin admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:20:12.159285 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7fd62a-cb46-488d-9414-6e122ccdef95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:20:12.159586 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7fd62a-cb46-488d-9414-6e122ccdef95 admin admin] PATH_INFO: `/v3/projects/e659f9b066994498979f08556383526f/users/fe3b0e80c70b4924bcfd73a4069cd0e6/roles/472313ae69174578824df0bed6390744` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:20:12.174567 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7fd62a-cb46-488d-9414-6e122ccdef95 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e659f9b066994498979f08556383526f, user_id=fe3b0e80c70b4924bcfd73a4069cd0e6, role_id=472313ae69174578824df0bed6390744)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:20:12.177293 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7fd62a-cb46-488d-9414-6e122ccdef95 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:20:12.191151 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1155/2307] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Jul 25 12:20:12 2022] PUT /identity/v3/projects/e659f9b066994498979f08556383526f/users/fe3b0e80c70b4924bcfd73a4069cd0e6/roles/472313ae69174578824df0bed6390744 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:20:12.203067 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce4ca9f-0219-40f4-813d-27261e37a316 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:20:12.203359 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce4ca9f-0219-40f4-813d-27261e37a316 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:20:12.203683 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce4ca9f-0219-40f4-813d-27261e37a316 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:20:12.232158 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-3ce4ca9f-0219-40f4-813d-27261e37a316 None None] MFA Rules not processed for user `fe3b0e80c70b4924bcfd73a4069cd0e6`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:20:12.233411 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-3ce4ca9f-0219-40f4-813d-27261e37a316 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:20:12.299482 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1153/2308] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:20:12 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:20:12.304097 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7fd62a-cb46-488d-9414-6e122ccdef95 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:20:12.369903 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7fd62a-cb46-488d-9414-6e122ccdef95 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:20:12.374517 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dadc5c5-6330-43e1-a24d-79b7f378b4f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:20:12.375170 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dadc5c5-6330-43e1-a24d-79b7f378b4f7 admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:20:12.375457 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dadc5c5-6330-43e1-a24d-79b7f378b4f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:20:12.375779 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dadc5c5-6330-43e1-a24d-79b7f378b4f7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:20:12.376474 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dadc5c5-6330-43e1-a24d-79b7f378b4f7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:20:12.378392 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dadc5c5-6330-43e1-a24d-79b7f378b4f7 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:20:12.408101 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1156/2309] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Jul 25 12:20:12 2022] POST /identity/v3/projects => generated 377 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:20:12.412088 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce4ca9f-0219-40f4-813d-27261e37a316 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:20:12.422724 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce4ca9f-0219-40f4-813d-27261e37a316 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:20:12.454802 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:20:12.481080 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76649bc-8ba6-4e8d-b434-bf85568c1f2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:20:12.481758 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76649bc-8ba6-4e8d-b434-bf85568c1f2f admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:20:12.482047 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76649bc-8ba6-4e8d-b434-bf85568c1f2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:20:12.482330 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76649bc-8ba6-4e8d-b434-bf85568c1f2f admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:20:12.483100 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76649bc-8ba6-4e8d-b434-bf85568c1f2f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:20:12.485076 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76649bc-8ba6-4e8d-b434-bf85568c1f2f admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:20:12.507515 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1154/2310] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Jul 25 12:20:12 2022] POST /identity/v3/users => generated 295 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 12:20:12.512629 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dadc5c5-6330-43e1-a24d-79b7f378b4f7 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:20:12.527093 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dadc5c5-6330-43e1-a24d-79b7f378b4f7 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:20:12.532019 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eabe57c-1194-4119-82f7-1c514ec4b5c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:20:12.532815 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eabe57c-1194-4119-82f7-1c514ec4b5c0 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:20:12.533091 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eabe57c-1194-4119-82f7-1c514ec4b5c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:20:12.533375 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eabe57c-1194-4119-82f7-1c514ec4b5c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:20:12.534109 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eabe57c-1194-4119-82f7-1c514ec4b5c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:20:12.535746 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eabe57c-1194-4119-82f7-1c514ec4b5c0 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:20:12.547297 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1157/2311] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Jul 25 12:20:12 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:20:12.551745 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76649bc-8ba6-4e8d-b434-bf85568c1f2f admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:20:12.564401 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76649bc-8ba6-4e8d-b434-bf85568c1f2f admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:20:12.569255 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e9f093-5f4e-4ee0-a9ea-7cfa9e06b818 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:20:12.570197 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e9f093-5f4e-4ee0-a9ea-7cfa9e06b818 admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:20:12.570483 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e9f093-5f4e-4ee0-a9ea-7cfa9e06b818 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:20:12.571617 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e9f093-5f4e-4ee0-a9ea-7cfa9e06b818 admin admin] PATH_INFO: `/v3/projects/7aaf707b83d34f3bb5dcf2e7ec8e0aa1/users/0d8b40ed2bf249df8bd0aca6c5696538/roles/a7d20885b6d94928a34b3cc5b3a00006` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:20:12.588503 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e9f093-5f4e-4ee0-a9ea-7cfa9e06b818 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7aaf707b83d34f3bb5dcf2e7ec8e0aa1, user_id=0d8b40ed2bf249df8bd0aca6c5696538, role_id=a7d20885b6d94928a34b3cc5b3a00006)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:20:12.591235 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e9f093-5f4e-4ee0-a9ea-7cfa9e06b818 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:20:12.609619 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1155/2312] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Jul 25 12:20:12 2022] PUT /identity/v3/projects/7aaf707b83d34f3bb5dcf2e7ec8e0aa1/users/0d8b40ed2bf249df8bd0aca6c5696538/roles/a7d20885b6d94928a34b3cc5b3a00006 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:20:12.611154 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eabe57c-1194-4119-82f7-1c514ec4b5c0 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:20:12.624126 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eabe57c-1194-4119-82f7-1c514ec4b5c0 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:20:12.630043 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:20:12.698951 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6e4d87-3997-4c2a-b523-5d649bd7c30e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:20:12.699798 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6e4d87-3997-4c2a-b523-5d649bd7c30e admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:20:12.700076 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6e4d87-3997-4c2a-b523-5d649bd7c30e admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:20:12.700355 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6e4d87-3997-4c2a-b523-5d649bd7c30e admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:20:12.701099 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6e4d87-3997-4c2a-b523-5d649bd7c30e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:20:12.702722 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6e4d87-3997-4c2a-b523-5d649bd7c30e admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:20:12.718237 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1158/2313] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Jul 25 12:20:12 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:20:12.719948 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e9f093-5f4e-4ee0-a9ea-7cfa9e06b818 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:20:12.731481 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e9f093-5f4e-4ee0-a9ea-7cfa9e06b818 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:20:12.736373 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86511c4a-9a18-47aa-b517-b9bd9783cd3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:20:12.737368 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86511c4a-9a18-47aa-b517-b9bd9783cd3d admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:20:12.737659 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86511c4a-9a18-47aa-b517-b9bd9783cd3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:20:12.737972 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86511c4a-9a18-47aa-b517-b9bd9783cd3d admin admin] PATH_INFO: `/v3/projects/7aaf707b83d34f3bb5dcf2e7ec8e0aa1/users/0d8b40ed2bf249df8bd0aca6c5696538/roles/a7d20885b6d94928a34b3cc5b3a00006` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:20:12.740265 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86511c4a-9a18-47aa-b517-b9bd9783cd3d admin admin] RBAC: Authorizing `identity:create_grant(project_id=7aaf707b83d34f3bb5dcf2e7ec8e0aa1, user_id=0d8b40ed2bf249df8bd0aca6c5696538, role_id=a7d20885b6d94928a34b3cc5b3a00006)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:20:12.742988 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86511c4a-9a18-47aa-b517-b9bd9783cd3d admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:20:12.754462 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1156/2314] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Jul 25 12:20:12 2022] PUT /identity/v3/projects/7aaf707b83d34f3bb5dcf2e7ec8e0aa1/users/0d8b40ed2bf249df8bd0aca6c5696538/roles/a7d20885b6d94928a34b3cc5b3a00006 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:20:12.765345 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21c8455-0a38-40e8-80fc-04ad445ba7b6 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:20:12.765655 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21c8455-0a38-40e8-80fc-04ad445ba7b6 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:20:12.766065 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21c8455-0a38-40e8-80fc-04ad445ba7b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:20:12.812536 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-d21c8455-0a38-40e8-80fc-04ad445ba7b6 None None] MFA Rules not processed for user `0d8b40ed2bf249df8bd0aca6c5696538`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:20:12.813919 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-d21c8455-0a38-40e8-80fc-04ad445ba7b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:20:12.908881 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1159/2315] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:20:12 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:20:12.919508 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86511c4a-9a18-47aa-b517-b9bd9783cd3d admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:20:12.922887 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-86511c4a-9a18-47aa-b517-b9bd9783cd3d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:20:13.010554 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86511c4a-9a18-47aa-b517-b9bd9783cd3d admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:20:13.014576 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c490d5e2-760d-48f5-95e6-125343c51102 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f89ce6f0cf04373a414d927fa0f8a7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:20:13.015456 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c490d5e2-760d-48f5-95e6-125343c51102 service cinder] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:20:13.015946 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c490d5e2-760d-48f5-95e6-125343c51102 service cinder] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:20:13.016776 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c490d5e2-760d-48f5-95e6-125343c51102 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:20:13.098271 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c490d5e2-760d-48f5-95e6-125343c51102 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:20:13.100207 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c490d5e2-760d-48f5-95e6-125343c51102 service cinder] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:20:13.110625 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1157/2316] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:20:12 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:20:17.112014 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21c8455-0a38-40e8-80fc-04ad445ba7b6 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:20:17.113525 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-d21c8455-0a38-40e8-80fc-04ad445ba7b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:20:17.197470 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21c8455-0a38-40e8-80fc-04ad445ba7b6 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:20:17.203035 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b396b8-572b-468f-877e-9e0c7d95b94b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f67142a1193d47c7bd33d7040268233c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:20:17.203730 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b396b8-572b-468f-877e-9e0c7d95b94b service nova] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:20:17.204021 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b396b8-572b-468f-877e-9e0c7d95b94b service nova] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:20:17.204307 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b396b8-572b-468f-877e-9e0c7d95b94b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:20:17.206116 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-e1b396b8-572b-468f-877e-9e0c7d95b94b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:20:17.233775 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:20:17.269467 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b396b8-572b-468f-877e-9e0c7d95b94b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:20:17.271054 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b396b8-572b-468f-877e-9e0c7d95b94b service nova] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:20:17.280684 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1160/2317] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:20:17 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:20:18.676999 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e731a6bb-8443-4107-8da6-23b8696ff326 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:20:18.677438 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e731a6bb-8443-4107-8da6-23b8696ff326 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:20:18.677563 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e731a6bb-8443-4107-8da6-23b8696ff326 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:20:18.698314 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-e731a6bb-8443-4107-8da6-23b8696ff326 None None] MFA Rules not processed for user `93207b858d294d049fa56d45810a40aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:20:18.699684 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-e731a6bb-8443-4107-8da6-23b8696ff326 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:20:18.740882 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:20:18.759648 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1158/2318] 10.222.0.42 () {58 vars in 1009 bytes} [Mon Jul 25 12:20:18 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:20:18.799236 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b396b8-572b-468f-877e-9e0c7d95b94b service nova] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:20:18.804302 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-e1b396b8-572b-468f-877e-9e0c7d95b94b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:20:18.827191 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b396b8-572b-468f-877e-9e0c7d95b94b service nova] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:20:18.831787 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b296a3d1-2a55-4dbd-83cb-5b9116ddd941 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f89ce6f0cf04373a414d927fa0f8a7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:20:18.832456 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b296a3d1-2a55-4dbd-83cb-5b9116ddd941 service cinder] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:20:18.832736 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b296a3d1-2a55-4dbd-83cb-5b9116ddd941 service cinder] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:20:18.833029 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b296a3d1-2a55-4dbd-83cb-5b9116ddd941 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:20:18.836095 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e731a6bb-8443-4107-8da6-23b8696ff326 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:20:18.837636 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-e731a6bb-8443-4107-8da6-23b8696ff326 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:20:18.848927 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b296a3d1-2a55-4dbd-83cb-5b9116ddd941 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:20:18.850601 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b296a3d1-2a55-4dbd-83cb-5b9116ddd941 service cinder] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:20:18.875177 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1161/2319] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:20:18 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 76 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:20:18.933708 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e731a6bb-8443-4107-8da6-23b8696ff326 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:20:18.941015 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4769b64e-c4a8-47cb-bf0a-0c304e6352a2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bd782ea66364eeb96592096eae63d4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:20:18.941473 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4769b64e-c4a8-47cb-bf0a-0c304e6352a2 service swift] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:20:18.941756 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4769b64e-c4a8-47cb-bf0a-0c304e6352a2 service swift] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:20:18.942146 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4769b64e-c4a8-47cb-bf0a-0c304e6352a2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:20:18.955252 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4769b64e-c4a8-47cb-bf0a-0c304e6352a2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:20:18.958718 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4769b64e-c4a8-47cb-bf0a-0c304e6352a2 service swift] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:20:18.992502 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1159/2320] 10.222.0.42 () {60 vars in 1384 bytes} [Mon Jul 25 12:20:18 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 25 12:20:28.857808 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=22 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:20:28.966603 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=38 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:20:33.827572 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b296a3d1-2a55-4dbd-83cb-5b9116ddd941 service cinder] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:20:33.829117 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-b296a3d1-2a55-4dbd-83cb-5b9116ddd941 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:20:33.849670 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b296a3d1-2a55-4dbd-83cb-5b9116ddd941 service cinder] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:20:33.855373 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f000e30-5842-42e9-891d-74c3e805153b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f67142a1193d47c7bd33d7040268233c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:20:33.856239 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f000e30-5842-42e9-891d-74c3e805153b service nova] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:20:33.856354 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f000e30-5842-42e9-891d-74c3e805153b service nova] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:20:33.856624 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f000e30-5842-42e9-891d-74c3e805153b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:20:33.859056 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-4f000e30-5842-42e9-891d-74c3e805153b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:20:33.871970 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f000e30-5842-42e9-891d-74c3e805153b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:20:33.873760 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f000e30-5842-42e9-891d-74c3e805153b service nova] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:20:33.887109 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1162/2321] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:20:33 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:20:34.037587 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f306e0c-5841-4d56-9f1c-e86f84abb4b4 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:20:34.038512 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f306e0c-5841-4d56-9f1c-e86f84abb4b4 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:20:34.038801 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f306e0c-5841-4d56-9f1c-e86f84abb4b4 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:20:34.042313 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1160/2322] 10.222.0.42 () {54 vars in 921 bytes} [Mon Jul 25 12:20:34 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:20:34.048407 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2be24a8-08ec-4500-80af-f74ce9fc6ed7 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:20:34.048781 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2be24a8-08ec-4500-80af-f74ce9fc6ed7 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:20:34.049022 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2be24a8-08ec-4500-80af-f74ce9fc6ed7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:20:34.073904 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-a2be24a8-08ec-4500-80af-f74ce9fc6ed7 None None] MFA Rules not processed for user `f67142a1193d47c7bd33d7040268233c`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:20:34.075511 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-a2be24a8-08ec-4500-80af-f74ce9fc6ed7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:20:34.086014 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1163/2323] 10.222.0.42 () {58 vars in 1018 bytes} [Mon Jul 25 12:20:34 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:20:34.098435 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f306e0c-5841-4d56-9f1c-e86f84abb4b4 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:20:34.100391 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-7f306e0c-5841-4d56-9f1c-e86f84abb4b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:20:34.167119 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f306e0c-5841-4d56-9f1c-e86f84abb4b4 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:20:34.170952 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0684dc-70f3-44ca-abaf-6d01da3fc0c4 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ea16528b1374bae9cf54603f7907bfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:20:34.171833 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0684dc-70f3-44ca-abaf-6d01da3fc0c4 service glance] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:20:34.172123 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0684dc-70f3-44ca-abaf-6d01da3fc0c4 service glance] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:20:34.172412 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0684dc-70f3-44ca-abaf-6d01da3fc0c4 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:20:34.184195 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0684dc-70f3-44ca-abaf-6d01da3fc0c4 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:20:34.185657 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0684dc-70f3-44ca-abaf-6d01da3fc0c4 service glance] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:20:34.199743 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1161/2324] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:20:34 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:20:34.267684 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c25d9b8-0722-46d4-825f-f7b71f7c9a20 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:20:34.267885 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c25d9b8-0722-46d4-825f-f7b71f7c9a20 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:20:34.268166 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c25d9b8-0722-46d4-825f-f7b71f7c9a20 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:20:34.269576 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1164/2325] 10.222.0.42 () {54 vars in 921 bytes} [Mon Jul 25 12:20:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:20:34.275392 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf46a46-d41c-463f-853b-f4982a2e6ad3 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:20:34.275730 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf46a46-d41c-463f-853b-f4982a2e6ad3 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:20:34.276016 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf46a46-d41c-463f-853b-f4982a2e6ad3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:20:34.296181 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-baf46a46-d41c-463f-853b-f4982a2e6ad3 None None] MFA Rules not processed for user `51733781201243e8abf9eca39b207750`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:20:34.297380 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-baf46a46-d41c-463f-853b-f4982a2e6ad3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:20:34.347138 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1162/2326] 10.222.0.42 () {58 vars in 1018 bytes} [Mon Jul 25 12:20:34 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:20:34.359660 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c25d9b8-0722-46d4-825f-f7b71f7c9a20 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:20:34.361359 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-2c25d9b8-0722-46d4-825f-f7b71f7c9a20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:20:34.374287 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c25d9b8-0722-46d4-825f-f7b71f7c9a20 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:20:34.377768 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b807de0a-421a-4330-bab4-85139043d3c5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51733781201243e8abf9eca39b207750', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:20:34.378401 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b807de0a-421a-4330-bab4-85139043d3c5 service neutron] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:20:34.378688 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b807de0a-421a-4330-bab4-85139043d3c5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:20:34.378970 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b807de0a-421a-4330-bab4-85139043d3c5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:20:34.385913 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b807de0a-421a-4330-bab4-85139043d3c5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:20:34.387325 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b807de0a-421a-4330-bab4-85139043d3c5 service neutron] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:20:34.395007 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1165/2327] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:20:34 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:20:34.475931 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d309c763-041a-43d2-9057-15f607c7a69d None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:20:34.476170 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d309c763-041a-43d2-9057-15f607c7a69d None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:20:34.476451 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d309c763-041a-43d2-9057-15f607c7a69d None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:20:34.478520 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1163/2328] 10.222.0.42 () {54 vars in 921 bytes} [Mon Jul 25 12:20:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 12:20:34.484124 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd402d96-e155-471d-8e6e-cac55329acf6 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:20:34.484419 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd402d96-e155-471d-8e6e-cac55329acf6 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:20:34.484704 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd402d96-e155-471d-8e6e-cac55329acf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:20:34.504845 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-cd402d96-e155-471d-8e6e-cac55329acf6 None None] MFA Rules not processed for user `f67142a1193d47c7bd33d7040268233c`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:20:34.506036 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-cd402d96-e155-471d-8e6e-cac55329acf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:20:34.515784 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1166/2329] 10.222.0.42 () {58 vars in 1018 bytes} [Mon Jul 25 12:20:34 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:20:34.526744 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d309c763-041a-43d2-9057-15f607c7a69d None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:20:34.538952 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d309c763-041a-43d2-9057-15f607c7a69d None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:20:34.543297 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d743eec0-17f8-42ba-a694-b04610387736 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51733781201243e8abf9eca39b207750', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:20:34.543973 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d743eec0-17f8-42ba-a694-b04610387736 service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:20:34.544260 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d743eec0-17f8-42ba-a694-b04610387736 service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:20:34.544541 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d743eec0-17f8-42ba-a694-b04610387736 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:20:34.554930 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d743eec0-17f8-42ba-a694-b04610387736 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:20:34.556457 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d743eec0-17f8-42ba-a694-b04610387736 service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:20:34.567144 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1164/2330] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:20:34 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:20:36.123233 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d7328d-5c11-43f6-bec3-d972dabe87d8 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:20:36.123732 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d7328d-5c11-43f6-bec3-d972dabe87d8 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:20:36.123846 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d7328d-5c11-43f6-bec3-d972dabe87d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:20:36.142737 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-c5d7328d-5c11-43f6-bec3-d972dabe87d8 None None] MFA Rules not processed for user `93207b858d294d049fa56d45810a40aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:20:36.143985 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-c5d7328d-5c11-43f6-bec3-d972dabe87d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:20:36.153110 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1167/2331] 10.222.0.42 () {58 vars in 1009 bytes} [Mon Jul 25 12:20:36 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:20:36.165842 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d743eec0-17f8-42ba-a694-b04610387736 service neutron] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:20:36.177091 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d743eec0-17f8-42ba-a694-b04610387736 service neutron] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:20:36.182659 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b288e5ab-4b5b-4089-aae3-f89e25ecee5c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bd782ea66364eeb96592096eae63d4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:20:36.183304 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b288e5ab-4b5b-4089-aae3-f89e25ecee5c service swift] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:20:36.183624 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b288e5ab-4b5b-4089-aae3-f89e25ecee5c service swift] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:20:36.183920 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b288e5ab-4b5b-4089-aae3-f89e25ecee5c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:20:36.201083 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b288e5ab-4b5b-4089-aae3-f89e25ecee5c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:20:36.202741 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b288e5ab-4b5b-4089-aae3-f89e25ecee5c service swift] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:20:36.211663 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1165/2332] 10.222.0.42 () {60 vars in 1384 bytes} [Mon Jul 25 12:20:36 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 25 12:20:44.408934 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d7328d-5c11-43f6-bec3-d972dabe87d8 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:20:44.480521 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d7328d-5c11-43f6-bec3-d972dabe87d8 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:20:44.485218 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b002e9-2b3c-4450-9239-37568e066c4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:20:44.486179 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b002e9-2b3c-4450-9239-37568e066c4b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:20:44.486533 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b002e9-2b3c-4450-9239-37568e066c4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:20:44.486878 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b002e9-2b3c-4450-9239-37568e066c4b admin admin] PATH_INFO: `/v3/users/fe3b0e80c70b4924bcfd73a4069cd0e6` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:20:44.488174 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b002e9-2b3c-4450-9239-37568e066c4b admin admin] RBAC: Authorizing `identity:delete_user(user_id=fe3b0e80c70b4924bcfd73a4069cd0e6)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:20:44.490173 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b002e9-2b3c-4450-9239-37568e066c4b admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:20:44.542370 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-70b002e9-2b3c-4450-9239-37568e066c4b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fe3b0e80c70b4924bcfd73a4069cd0e6'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:20:44.543026 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-70b002e9-2b3c-4450-9239-37568e066c4b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fe3b0e80c70b4924bcfd73a4069cd0e6'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:20:44.551459 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-70b002e9-2b3c-4450-9239-37568e066c4b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fe3b0e80c70b4924bcfd73a4069cd0e6'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:20:44.565035 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-70b002e9-2b3c-4450-9239-37568e066c4b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fe3b0e80c70b4924bcfd73a4069cd0e6'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:20:44.572648 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-70b002e9-2b3c-4450-9239-37568e066c4b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fe3b0e80c70b4924bcfd73a4069cd0e6'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:20:44.579061 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1168/2333] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Jul 25 12:20:44 2022] DELETE /identity/v3/users/fe3b0e80c70b4924bcfd73a4069cd0e6 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 12:20:44.583826 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b58e1cc-f880-446a-8e36-9093f115dafb None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:20:44.584331 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b58e1cc-f880-446a-8e36-9093f115dafb None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:20:44.584423 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b58e1cc-f880-446a-8e36-9093f115dafb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:20:44.605827 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-8b58e1cc-f880-446a-8e36-9093f115dafb None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:20:44.607303 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-8b58e1cc-f880-446a-8e36-9093f115dafb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:20:44.670278 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1166/2334] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:20:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:20:44.679909 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b002e9-2b3c-4450-9239-37568e066c4b admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:20:44.682052 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-70b002e9-2b3c-4450-9239-37568e066c4b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:20:44.720562 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:20:44.744949 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b002e9-2b3c-4450-9239-37568e066c4b admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:20:44.751681 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb857ae3-e26b-4046-86c5-86cdc91481f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51733781201243e8abf9eca39b207750', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:20:44.752462 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb857ae3-e26b-4046-86c5-86cdc91481f8 service neutron] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:20:44.752870 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb857ae3-e26b-4046-86c5-86cdc91481f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:20:44.753253 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb857ae3-e26b-4046-86c5-86cdc91481f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:20:44.764384 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb857ae3-e26b-4046-86c5-86cdc91481f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:20:44.765914 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb857ae3-e26b-4046-86c5-86cdc91481f8 service neutron] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:20:44.780236 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1169/2335] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:20:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:20:45.139249 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b58e1cc-f880-446a-8e36-9093f115dafb None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:20:45.141804 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-8b58e1cc-f880-446a-8e36-9093f115dafb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:20:45.156988 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b58e1cc-f880-446a-8e36-9093f115dafb None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:20:45.162520 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f571e4-4a11-4b3b-9414-242e259f7250 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:20:45.163597 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f571e4-4a11-4b3b-9414-242e259f7250 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:20:45.163817 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f571e4-4a11-4b3b-9414-242e259f7250 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:20:45.164098 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f571e4-4a11-4b3b-9414-242e259f7250 admin admin] PATH_INFO: `/v3/projects/e659f9b066994498979f08556383526f` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:20:45.165299 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f571e4-4a11-4b3b-9414-242e259f7250 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e659f9b066994498979f08556383526f)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:20:45.167780 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f571e4-4a11-4b3b-9414-242e259f7250 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:20:45.190837 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:20:45.213929 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-38f571e4-4a11-4b3b-9414-242e259f7250 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e659f9b066994498979f08556383526f', 'request_id': 'req-38f571e4-4a11-4b3b-9414-242e259f7250'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:20:45.222202 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-38f571e4-4a11-4b3b-9414-242e259f7250 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e659f9b066994498979f08556383526f', 'request_id': 'req-38f571e4-4a11-4b3b-9414-242e259f7250'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:20:45.229640 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-38f571e4-4a11-4b3b-9414-242e259f7250 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e659f9b066994498979f08556383526f', 'request_id': 'req-38f571e4-4a11-4b3b-9414-242e259f7250'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:20:45.241625 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-38f571e4-4a11-4b3b-9414-242e259f7250 admin admin] The token cache is being invalidate because project e659f9b066994498979f08556383526f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 12:20:45.241921 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-38f571e4-4a11-4b3b-9414-242e259f7250 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:20:45.242567 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-38f571e4-4a11-4b3b-9414-242e259f7250 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:20:45.244728 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1167/2336] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Jul 25 12:20:45 2022] DELETE /identity/v3/projects/e659f9b066994498979f08556383526f => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:20:45.248758 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb857ae3-e26b-4046-86c5-86cdc91481f8 service neutron] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:20:45.258316 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb857ae3-e26b-4046-86c5-86cdc91481f8 service neutron] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:20:45.314784 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252fec2b-f12e-441d-8d54-afd0adcee42a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:20:45.315677 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252fec2b-f12e-441d-8d54-afd0adcee42a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:20:45.315965 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252fec2b-f12e-441d-8d54-afd0adcee42a admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:20:45.316246 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252fec2b-f12e-441d-8d54-afd0adcee42a admin admin] PATH_INFO: `/v3/users/0d8b40ed2bf249df8bd0aca6c5696538` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:20:45.317425 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252fec2b-f12e-441d-8d54-afd0adcee42a admin admin] RBAC: Authorizing `identity:delete_user(user_id=0d8b40ed2bf249df8bd0aca6c5696538)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:20:45.319356 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252fec2b-f12e-441d-8d54-afd0adcee42a admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:20:45.365389 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-252fec2b-f12e-441d-8d54-afd0adcee42a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0d8b40ed2bf249df8bd0aca6c5696538'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:20:45.366047 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-252fec2b-f12e-441d-8d54-afd0adcee42a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0d8b40ed2bf249df8bd0aca6c5696538'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:20:45.373745 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-252fec2b-f12e-441d-8d54-afd0adcee42a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0d8b40ed2bf249df8bd0aca6c5696538'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:20:45.385802 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-252fec2b-f12e-441d-8d54-afd0adcee42a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0d8b40ed2bf249df8bd0aca6c5696538'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:20:45.392560 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-252fec2b-f12e-441d-8d54-afd0adcee42a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0d8b40ed2bf249df8bd0aca6c5696538'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:20:45.398437 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1170/2337] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Jul 25 12:20:45 2022] DELETE /identity/v3/users/0d8b40ed2bf249df8bd0aca6c5696538 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:20:45.676354 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f571e4-4a11-4b3b-9414-242e259f7250 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:20:45.677862 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-38f571e4-4a11-4b3b-9414-242e259f7250 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:20:45.691693 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f571e4-4a11-4b3b-9414-242e259f7250 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:20:45.755157 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f4da63-ba09-49ea-b4c9-4a7206fcbe78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:20:45.756051 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f4da63-ba09-49ea-b4c9-4a7206fcbe78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:20:45.756337 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f4da63-ba09-49ea-b4c9-4a7206fcbe78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:20:45.756617 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f4da63-ba09-49ea-b4c9-4a7206fcbe78 admin admin] PATH_INFO: `/v3/projects/7aaf707b83d34f3bb5dcf2e7ec8e0aa1` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:20:45.757811 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f4da63-ba09-49ea-b4c9-4a7206fcbe78 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7aaf707b83d34f3bb5dcf2e7ec8e0aa1)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:20:45.760415 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f4da63-ba09-49ea-b4c9-4a7206fcbe78 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:20:45.806892 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-05f4da63-ba09-49ea-b4c9-4a7206fcbe78 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7aaf707b83d34f3bb5dcf2e7ec8e0aa1', 'request_id': 'req-05f4da63-ba09-49ea-b4c9-4a7206fcbe78'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:20:45.813969 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-05f4da63-ba09-49ea-b4c9-4a7206fcbe78 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7aaf707b83d34f3bb5dcf2e7ec8e0aa1', 'request_id': 'req-05f4da63-ba09-49ea-b4c9-4a7206fcbe78'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:20:45.819463 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-05f4da63-ba09-49ea-b4c9-4a7206fcbe78 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7aaf707b83d34f3bb5dcf2e7ec8e0aa1', 'request_id': 'req-05f4da63-ba09-49ea-b4c9-4a7206fcbe78'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:20:45.830294 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-05f4da63-ba09-49ea-b4c9-4a7206fcbe78 admin admin] The token cache is being invalidate because project 7aaf707b83d34f3bb5dcf2e7ec8e0aa1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 12:20:45.830583 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-05f4da63-ba09-49ea-b4c9-4a7206fcbe78 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:20:45.831220 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-05f4da63-ba09-49ea-b4c9-4a7206fcbe78 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:20:45.832810 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1168/2338] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Jul 25 12:20:45 2022] DELETE /identity/v3/projects/7aaf707b83d34f3bb5dcf2e7ec8e0aa1 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:20:45.837644 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7fc544-97a6-4864-ab6f-5935c3812cd3 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:20:45.838121 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7fc544-97a6-4864-ab6f-5935c3812cd3 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:20:45.838240 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7fc544-97a6-4864-ab6f-5935c3812cd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:20:45.858004 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-1c7fc544-97a6-4864-ab6f-5935c3812cd3 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:20:45.859217 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-1c7fc544-97a6-4864-ab6f-5935c3812cd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:20:45.906172 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:20:45.919789 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1171/2339] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:20:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:20:45.928801 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9177aab0-ccb1-4795-a600-22604d7591b1 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:20:45.929094 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9177aab0-ccb1-4795-a600-22604d7591b1 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:20:45.929376 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9177aab0-ccb1-4795-a600-22604d7591b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:20:45.949793 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-9177aab0-ccb1-4795-a600-22604d7591b1 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:20:45.950926 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-9177aab0-ccb1-4795-a600-22604d7591b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:20:45.958726 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1169/2340] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:20:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:20:45.963450 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7fc544-97a6-4864-ab6f-5935c3812cd3 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:20:45.973084 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7fc544-97a6-4864-ab6f-5935c3812cd3 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:20:45.977659 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6aa646-5368-4c3d-b9cc-f3a3109ccc45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:20:45.978359 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6aa646-5368-4c3d-b9cc-f3a3109ccc45 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:20:45.978651 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6aa646-5368-4c3d-b9cc-f3a3109ccc45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:20:45.978935 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6aa646-5368-4c3d-b9cc-f3a3109ccc45 admin admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:20:45.979706 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6aa646-5368-4c3d-b9cc-f3a3109ccc45 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:20:45.980023 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6aa646-5368-4c3d-b9cc-f3a3109ccc45 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:20:45.981676 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6aa646-5368-4c3d-b9cc-f3a3109ccc45 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:20:45.993373 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1172/2341] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Jul 25 12:20:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:20:45.997803 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9177aab0-ccb1-4795-a600-22604d7591b1 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:20:46.007089 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9177aab0-ccb1-4795-a600-22604d7591b1 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:20:46.012164 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97e10d9-98ec-4870-8d4d-98944b631db4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:20:46.012817 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97e10d9-98ec-4870-8d4d-98944b631db4 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:20:46.013106 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97e10d9-98ec-4870-8d4d-98944b631db4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:20:46.013387 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97e10d9-98ec-4870-8d4d-98944b631db4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:20:46.014198 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97e10d9-98ec-4870-8d4d-98944b631db4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:20:46.016140 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97e10d9-98ec-4870-8d4d-98944b631db4 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:20:46.047754 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1170/2342] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Jul 25 12:20:45 2022] POST /identity/v3/projects => generated 369 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:20:46.049225 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6aa646-5368-4c3d-b9cc-f3a3109ccc45 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:20:46.059708 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6aa646-5368-4c3d-b9cc-f3a3109ccc45 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:20:46.117273 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3264fa8c-20e0-46fa-ac46-c6c2a4cfb9d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:20:46.117948 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3264fa8c-20e0-46fa-ac46-c6c2a4cfb9d5 admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:20:46.118240 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3264fa8c-20e0-46fa-ac46-c6c2a4cfb9d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:20:46.118707 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3264fa8c-20e0-46fa-ac46-c6c2a4cfb9d5 admin admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:20:46.119244 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3264fa8c-20e0-46fa-ac46-c6c2a4cfb9d5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:20:46.121240 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3264fa8c-20e0-46fa-ac46-c6c2a4cfb9d5 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:20:46.147254 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1173/2343] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Jul 25 12:20:46 2022] POST /identity/v3/users => generated 285 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:20:46.148517 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97e10d9-98ec-4870-8d4d-98944b631db4 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:20:46.159776 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97e10d9-98ec-4870-8d4d-98944b631db4 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:20:46.164369 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233ffc24-a7cb-4048-ab4e-c73b97bb0d63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:20:46.165029 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233ffc24-a7cb-4048-ab4e-c73b97bb0d63 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:20:46.165318 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233ffc24-a7cb-4048-ab4e-c73b97bb0d63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:20:46.165627 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233ffc24-a7cb-4048-ab4e-c73b97bb0d63 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:20:46.166360 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233ffc24-a7cb-4048-ab4e-c73b97bb0d63 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:20:46.168066 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233ffc24-a7cb-4048-ab4e-c73b97bb0d63 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:20:46.173172 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:DELETE=14,keystone:SELECT=83,keystone:INSERT=3 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:20:46.179491 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1171/2344] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Jul 25 12:20:46 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:20:46.184118 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3264fa8c-20e0-46fa-ac46-c6c2a4cfb9d5 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:20:46.194155 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3264fa8c-20e0-46fa-ac46-c6c2a4cfb9d5 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:20:46.199428 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db96ce97-5669-490d-8b41-d40d38dcf628 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:20:46.200165 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db96ce97-5669-490d-8b41-d40d38dcf628 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:20:46.200405 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db96ce97-5669-490d-8b41-d40d38dcf628 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:20:46.200688 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db96ce97-5669-490d-8b41-d40d38dcf628 admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:20:46.201417 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db96ce97-5669-490d-8b41-d40d38dcf628 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:20:46.203017 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db96ce97-5669-490d-8b41-d40d38dcf628 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:20:46.213498 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1174/2345] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Jul 25 12:20:46 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 12:20:46.217650 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233ffc24-a7cb-4048-ab4e-c73b97bb0d63 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:20:46.227603 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233ffc24-a7cb-4048-ab4e-c73b97bb0d63 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:20:46.232634 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bab673f-8995-4271-9bff-f261cdbe4577 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:20:46.233508 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bab673f-8995-4271-9bff-f261cdbe4577 admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:20:46.233795 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bab673f-8995-4271-9bff-f261cdbe4577 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:20:46.234081 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bab673f-8995-4271-9bff-f261cdbe4577 admin admin] PATH_INFO: `/v3/projects/53663516681d41d8a9fdd1f132b66581/users/42394bbf038e402bad87580c1c2ed5dd/roles/472313ae69174578824df0bed6390744` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:20:46.249040 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bab673f-8995-4271-9bff-f261cdbe4577 admin admin] RBAC: Authorizing `identity:create_grant(project_id=53663516681d41d8a9fdd1f132b66581, user_id=42394bbf038e402bad87580c1c2ed5dd, role_id=472313ae69174578824df0bed6390744)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:20:46.251732 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bab673f-8995-4271-9bff-f261cdbe4577 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:20:46.264796 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1172/2346] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Jul 25 12:20:46 2022] PUT /identity/v3/projects/53663516681d41d8a9fdd1f132b66581/users/42394bbf038e402bad87580c1c2ed5dd/roles/472313ae69174578824df0bed6390744 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:20:46.275142 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd4ca17-8105-44aa-9f9e-4955b7995922 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:20:46.275508 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd4ca17-8105-44aa-9f9e-4955b7995922 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:20:46.276132 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd4ca17-8105-44aa-9f9e-4955b7995922 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:20:46.303907 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-5dd4ca17-8105-44aa-9f9e-4955b7995922 None None] MFA Rules not processed for user `42394bbf038e402bad87580c1c2ed5dd`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:20:46.305131 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-5dd4ca17-8105-44aa-9f9e-4955b7995922 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:20:46.369728 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1175/2347] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:20:46 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:20:46.374298 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bab673f-8995-4271-9bff-f261cdbe4577 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:20:46.434932 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bab673f-8995-4271-9bff-f261cdbe4577 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:20:46.439629 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fbb6f1-a1b4-4612-bab1-286fffa9f3a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:20:46.440283 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fbb6f1-a1b4-4612-bab1-286fffa9f3a6 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:20:46.440572 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fbb6f1-a1b4-4612-bab1-286fffa9f3a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:20:46.440848 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fbb6f1-a1b4-4612-bab1-286fffa9f3a6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:20:46.441538 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42fbb6f1-a1b4-4612-bab1-286fffa9f3a6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:20:46.443458 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42fbb6f1-a1b4-4612-bab1-286fffa9f3a6 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:20:46.478141 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1173/2348] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Jul 25 12:20:46 2022] POST /identity/v3/projects => generated 369 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:20:46.482080 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd4ca17-8105-44aa-9f9e-4955b7995922 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:20:46.493082 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd4ca17-8105-44aa-9f9e-4955b7995922 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:20:46.541998 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:20:46.552754 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9330e7-c6ef-4cc3-948b-916d58e8e331 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:20:46.553428 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9330e7-c6ef-4cc3-948b-916d58e8e331 admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:20:46.553716 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9330e7-c6ef-4cc3-948b-916d58e8e331 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:20:46.554000 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9330e7-c6ef-4cc3-948b-916d58e8e331 admin admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:20:46.554704 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9330e7-c6ef-4cc3-948b-916d58e8e331 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:20:46.557283 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9330e7-c6ef-4cc3-948b-916d58e8e331 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:20:46.578441 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1176/2349] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Jul 25 12:20:46 2022] POST /identity/v3/users => generated 291 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:20:46.582452 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fbb6f1-a1b4-4612-bab1-286fffa9f3a6 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:20:46.592833 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fbb6f1-a1b4-4612-bab1-286fffa9f3a6 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:20:46.597494 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ec98b7-4dea-4834-94fd-d9be0bc3f987 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:20:46.598153 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ec98b7-4dea-4834-94fd-d9be0bc3f987 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:20:46.598445 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ec98b7-4dea-4834-94fd-d9be0bc3f987 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:20:46.598732 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ec98b7-4dea-4834-94fd-d9be0bc3f987 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:20:46.599464 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ec98b7-4dea-4834-94fd-d9be0bc3f987 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:20:46.601094 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ec98b7-4dea-4834-94fd-d9be0bc3f987 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:20:46.612323 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1174/2350] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Jul 25 12:20:46 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:20:46.616296 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9330e7-c6ef-4cc3-948b-916d58e8e331 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:20:46.626865 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9330e7-c6ef-4cc3-948b-916d58e8e331 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:20:46.631718 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be84e77f-7bc4-4392-979f-a6030e0757f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:20:46.632595 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be84e77f-7bc4-4392-979f-a6030e0757f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:20:46.632881 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be84e77f-7bc4-4392-979f-a6030e0757f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:20:46.633159 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be84e77f-7bc4-4392-979f-a6030e0757f7 admin admin] PATH_INFO: `/v3/projects/43c025269250448da52bfa23d42f6c8c/users/5ea4ebdceff6402fb4bcc614c1b6c6d3/roles/a7d20885b6d94928a34b3cc5b3a00006` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:20:46.647437 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be84e77f-7bc4-4392-979f-a6030e0757f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=43c025269250448da52bfa23d42f6c8c, user_id=5ea4ebdceff6402fb4bcc614c1b6c6d3, role_id=a7d20885b6d94928a34b3cc5b3a00006)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:20:46.650117 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be84e77f-7bc4-4392-979f-a6030e0757f7 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:20:46.662617 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1177/2351] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Jul 25 12:20:46 2022] PUT /identity/v3/projects/43c025269250448da52bfa23d42f6c8c/users/5ea4ebdceff6402fb4bcc614c1b6c6d3/roles/a7d20885b6d94928a34b3cc5b3a00006 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 12:20:46.667009 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ec98b7-4dea-4834-94fd-d9be0bc3f987 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:20:46.676996 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ec98b7-4dea-4834-94fd-d9be0bc3f987 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:20:46.733355 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ee4446-7921-4355-86c1-28658ce4c805 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:20:46.733958 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ee4446-7921-4355-86c1-28658ce4c805 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:20:46.734253 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ee4446-7921-4355-86c1-28658ce4c805 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:20:46.734536 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ee4446-7921-4355-86c1-28658ce4c805 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:20:46.735328 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ee4446-7921-4355-86c1-28658ce4c805 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:20:46.736978 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ee4446-7921-4355-86c1-28658ce4c805 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:20:46.747329 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1175/2352] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Jul 25 12:20:46 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:20:46.751456 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be84e77f-7bc4-4392-979f-a6030e0757f7 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:20:46.761782 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be84e77f-7bc4-4392-979f-a6030e0757f7 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:20:46.766675 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805b49fe-cd59-4b6e-a8b7-eef27f3056ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:20:46.767974 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805b49fe-cd59-4b6e-a8b7-eef27f3056ff admin admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:20:46.768288 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805b49fe-cd59-4b6e-a8b7-eef27f3056ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:20:46.768571 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805b49fe-cd59-4b6e-a8b7-eef27f3056ff admin admin] PATH_INFO: `/v3/projects/43c025269250448da52bfa23d42f6c8c/users/5ea4ebdceff6402fb4bcc614c1b6c6d3/roles/a7d20885b6d94928a34b3cc5b3a00006` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:20:46.770649 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805b49fe-cd59-4b6e-a8b7-eef27f3056ff admin admin] RBAC: Authorizing `identity:create_grant(project_id=43c025269250448da52bfa23d42f6c8c, user_id=5ea4ebdceff6402fb4bcc614c1b6c6d3, role_id=a7d20885b6d94928a34b3cc5b3a00006)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:20:46.773722 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805b49fe-cd59-4b6e-a8b7-eef27f3056ff admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:20:46.784841 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1178/2353] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Jul 25 12:20:46 2022] PUT /identity/v3/projects/43c025269250448da52bfa23d42f6c8c/users/5ea4ebdceff6402fb4bcc614c1b6c6d3/roles/a7d20885b6d94928a34b3cc5b3a00006 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:20:46.795488 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e731f32-6de7-48c3-a693-4b77698ef363 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:20:46.795826 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e731f32-6de7-48c3-a693-4b77698ef363 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:20:46.796107 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e731f32-6de7-48c3-a693-4b77698ef363 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:20:46.823803 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-9e731f32-6de7-48c3-a693-4b77698ef363 None None] MFA Rules not processed for user `5ea4ebdceff6402fb4bcc614c1b6c6d3`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:20:46.825030 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-9e731f32-6de7-48c3-a693-4b77698ef363 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:20:46.857788 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:20:46.893057 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1176/2354] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:20:46 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:20:46.905038 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805b49fe-cd59-4b6e-a8b7-eef27f3056ff admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:20:46.906909 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-805b49fe-cd59-4b6e-a8b7-eef27f3056ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:20:46.970101 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805b49fe-cd59-4b6e-a8b7-eef27f3056ff admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:20:46.973826 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e34596a-ac58-4a51-8165-4c9eaa0351fd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f89ce6f0cf04373a414d927fa0f8a7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:20:46.974482 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e34596a-ac58-4a51-8165-4c9eaa0351fd service cinder] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:20:46.974774 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e34596a-ac58-4a51-8165-4c9eaa0351fd service cinder] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:20:46.975059 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e34596a-ac58-4a51-8165-4c9eaa0351fd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:20:46.984881 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e34596a-ac58-4a51-8165-4c9eaa0351fd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:20:46.986248 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e34596a-ac58-4a51-8165-4c9eaa0351fd service cinder] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:20:46.995948 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1179/2355] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:20:46 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:20:47.045642 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e731f32-6de7-48c3-a693-4b77698ef363 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:20:47.047065 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-9e731f32-6de7-48c3-a693-4b77698ef363 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:20:47.062080 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e731f32-6de7-48c3-a693-4b77698ef363 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:20:47.065994 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7daf9c0-0136-4725-ac4e-57429a03c233 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f89ce6f0cf04373a414d927fa0f8a7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:20:47.066652 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7daf9c0-0136-4725-ac4e-57429a03c233 service cinder] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:20:47.066959 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7daf9c0-0136-4725-ac4e-57429a03c233 service cinder] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:20:47.067221 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7daf9c0-0136-4725-ac4e-57429a03c233 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:20:47.129383 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7daf9c0-0136-4725-ac4e-57429a03c233 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:20:47.131245 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7daf9c0-0136-4725-ac4e-57429a03c233 service cinder] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:20:47.143096 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1177/2356] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:20:47 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:20:50.508737 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e34596a-ac58-4a51-8165-4c9eaa0351fd service cinder] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:20:50.584032 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e34596a-ac58-4a51-8165-4c9eaa0351fd service cinder] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:20:50.588804 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecaec67-7b6b-416d-b3c8-c06c21981967 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:20:50.589694 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecaec67-7b6b-416d-b3c8-c06c21981967 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:20:50.589980 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecaec67-7b6b-416d-b3c8-c06c21981967 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:20:50.590260 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecaec67-7b6b-416d-b3c8-c06c21981967 admin admin] PATH_INFO: `/v3/users/42394bbf038e402bad87580c1c2ed5dd` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:20:50.591466 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecaec67-7b6b-416d-b3c8-c06c21981967 admin admin] RBAC: Authorizing `identity:delete_user(user_id=42394bbf038e402bad87580c1c2ed5dd)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:20:50.593459 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecaec67-7b6b-416d-b3c8-c06c21981967 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:20:50.639623 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-9ecaec67-7b6b-416d-b3c8-c06c21981967 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '42394bbf038e402bad87580c1c2ed5dd'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:20:50.640301 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-9ecaec67-7b6b-416d-b3c8-c06c21981967 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '42394bbf038e402bad87580c1c2ed5dd'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:20:50.641955 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:20:50.647865 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-9ecaec67-7b6b-416d-b3c8-c06c21981967 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '42394bbf038e402bad87580c1c2ed5dd'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:20:50.661629 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-9ecaec67-7b6b-416d-b3c8-c06c21981967 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '42394bbf038e402bad87580c1c2ed5dd'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:20:50.669033 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-9ecaec67-7b6b-416d-b3c8-c06c21981967 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '42394bbf038e402bad87580c1c2ed5dd'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:20:50.675014 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1180/2357] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Jul 25 12:20:50 2022] DELETE /identity/v3/users/42394bbf038e402bad87580c1c2ed5dd => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:20:50.679455 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ee8f8b-575c-48d6-b74a-cd3f56b231e6 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:20:50.679928 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ee8f8b-575c-48d6-b74a-cd3f56b231e6 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:20:50.680048 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ee8f8b-575c-48d6-b74a-cd3f56b231e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:20:50.698819 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-42ee8f8b-575c-48d6-b74a-cd3f56b231e6 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:20:50.700065 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-42ee8f8b-575c-48d6-b74a-cd3f56b231e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:20:50.757669 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1178/2358] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:20:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:20:50.767095 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecaec67-7b6b-416d-b3c8-c06c21981967 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:20:50.768556 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-9ecaec67-7b6b-416d-b3c8-c06c21981967 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:20:50.826284 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecaec67-7b6b-416d-b3c8-c06c21981967 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:20:50.829958 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971debd5-52d5-41fc-a506-65a12684cf0a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51733781201243e8abf9eca39b207750', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:20:50.830614 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971debd5-52d5-41fc-a506-65a12684cf0a service neutron] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:20:50.830981 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971debd5-52d5-41fc-a506-65a12684cf0a service neutron] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:20:50.831174 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971debd5-52d5-41fc-a506-65a12684cf0a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:20:50.841237 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971debd5-52d5-41fc-a506-65a12684cf0a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:20:50.842704 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971debd5-52d5-41fc-a506-65a12684cf0a service neutron] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:20:50.853377 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1181/2359] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:20:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:20:51.205435 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ee8f8b-575c-48d6-b74a-cd3f56b231e6 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:20:51.207284 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-42ee8f8b-575c-48d6-b74a-cd3f56b231e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:20:51.222330 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ee8f8b-575c-48d6-b74a-cd3f56b231e6 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:20:51.230530 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae0994e-227e-425b-9ec5-aba66ef2778b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:20:51.232115 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae0994e-227e-425b-9ec5-aba66ef2778b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:20:51.232652 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae0994e-227e-425b-9ec5-aba66ef2778b admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:20:51.233166 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae0994e-227e-425b-9ec5-aba66ef2778b admin admin] PATH_INFO: `/v3/projects/53663516681d41d8a9fdd1f132b66581` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:20:51.234665 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae0994e-227e-425b-9ec5-aba66ef2778b admin admin] RBAC: Authorizing `identity:delete_project(project_id=53663516681d41d8a9fdd1f132b66581)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:20:51.236836 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae0994e-227e-425b-9ec5-aba66ef2778b admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:20:51.289728 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-9ae0994e-227e-425b-9ec5-aba66ef2778b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '53663516681d41d8a9fdd1f132b66581', 'request_id': 'req-9ae0994e-227e-425b-9ec5-aba66ef2778b'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:20:51.297450 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-9ae0994e-227e-425b-9ec5-aba66ef2778b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '53663516681d41d8a9fdd1f132b66581', 'request_id': 'req-9ae0994e-227e-425b-9ec5-aba66ef2778b'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:20:51.303162 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-9ae0994e-227e-425b-9ec5-aba66ef2778b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '53663516681d41d8a9fdd1f132b66581', 'request_id': 'req-9ae0994e-227e-425b-9ec5-aba66ef2778b'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:20:51.304269 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:20:51.315597 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-9ae0994e-227e-425b-9ec5-aba66ef2778b admin admin] The token cache is being invalidate because project 53663516681d41d8a9fdd1f132b66581 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 12:20:51.316107 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-9ae0994e-227e-425b-9ec5-aba66ef2778b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:20:51.316875 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-9ae0994e-227e-425b-9ec5-aba66ef2778b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:20:51.318758 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1179/2360] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Jul 25 12:20:51 2022] DELETE /identity/v3/projects/53663516681d41d8a9fdd1f132b66581 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:20:51.323466 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971debd5-52d5-41fc-a506-65a12684cf0a service neutron] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:20:51.333270 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971debd5-52d5-41fc-a506-65a12684cf0a service neutron] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:20:51.389268 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5492226-d917-4037-b7a0-8192ce6ab0f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:20:51.390265 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5492226-d917-4037-b7a0-8192ce6ab0f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:20:51.390589 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5492226-d917-4037-b7a0-8192ce6ab0f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:20:51.390877 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5492226-d917-4037-b7a0-8192ce6ab0f9 admin admin] PATH_INFO: `/v3/users/5ea4ebdceff6402fb4bcc614c1b6c6d3` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:20:51.392122 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5492226-d917-4037-b7a0-8192ce6ab0f9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5ea4ebdceff6402fb4bcc614c1b6c6d3)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:20:51.394048 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5492226-d917-4037-b7a0-8192ce6ab0f9 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:20:51.439820 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-c5492226-d917-4037-b7a0-8192ce6ab0f9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5ea4ebdceff6402fb4bcc614c1b6c6d3'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:20:51.440492 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-c5492226-d917-4037-b7a0-8192ce6ab0f9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5ea4ebdceff6402fb4bcc614c1b6c6d3'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:20:51.448468 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-c5492226-d917-4037-b7a0-8192ce6ab0f9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5ea4ebdceff6402fb4bcc614c1b6c6d3'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:20:51.459498 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:INSERT=2,keystone:DELETE=13,keystone:SELECT=85 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:20:51.460790 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-c5492226-d917-4037-b7a0-8192ce6ab0f9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5ea4ebdceff6402fb4bcc614c1b6c6d3'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:20:51.469736 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-c5492226-d917-4037-b7a0-8192ce6ab0f9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5ea4ebdceff6402fb4bcc614c1b6c6d3'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:20:51.475593 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1182/2361] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Jul 25 12:20:51 2022] DELETE /identity/v3/users/5ea4ebdceff6402fb4bcc614c1b6c6d3 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:20:51.767031 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae0994e-227e-425b-9ec5-aba66ef2778b admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:20:51.768644 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-9ae0994e-227e-425b-9ec5-aba66ef2778b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:20:51.793385 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae0994e-227e-425b-9ec5-aba66ef2778b admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:20:51.866805 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39987e46-db87-4cbe-9f72-e76340cecde1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:20:51.867785 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39987e46-db87-4cbe-9f72-e76340cecde1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:20:51.868186 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39987e46-db87-4cbe-9f72-e76340cecde1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:20:51.869384 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39987e46-db87-4cbe-9f72-e76340cecde1 admin admin] PATH_INFO: `/v3/projects/43c025269250448da52bfa23d42f6c8c` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:20:51.870827 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39987e46-db87-4cbe-9f72-e76340cecde1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=43c025269250448da52bfa23d42f6c8c)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:20:51.873767 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39987e46-db87-4cbe-9f72-e76340cecde1 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:20:51.931172 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-39987e46-db87-4cbe-9f72-e76340cecde1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '43c025269250448da52bfa23d42f6c8c', 'request_id': 'req-39987e46-db87-4cbe-9f72-e76340cecde1'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:20:51.945391 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-39987e46-db87-4cbe-9f72-e76340cecde1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '43c025269250448da52bfa23d42f6c8c', 'request_id': 'req-39987e46-db87-4cbe-9f72-e76340cecde1'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:20:51.954493 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-39987e46-db87-4cbe-9f72-e76340cecde1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '43c025269250448da52bfa23d42f6c8c', 'request_id': 'req-39987e46-db87-4cbe-9f72-e76340cecde1'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:20:51.978214 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-39987e46-db87-4cbe-9f72-e76340cecde1 admin admin] The token cache is being invalidate because project 43c025269250448da52bfa23d42f6c8c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 12:20:51.978525 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-39987e46-db87-4cbe-9f72-e76340cecde1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:20:51.979194 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-39987e46-db87-4cbe-9f72-e76340cecde1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:20:51.981069 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1180/2362] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Jul 25 12:20:51 2022] DELETE /identity/v3/projects/43c025269250448da52bfa23d42f6c8c => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:20:51.985770 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e59272-157c-4b27-af56-0e57f03089ae None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:20:51.986581 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e59272-157c-4b27-af56-0e57f03089ae None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:20:51.987164 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e59272-157c-4b27-af56-0e57f03089ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:20:52.010361 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-a6e59272-157c-4b27-af56-0e57f03089ae None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:20:52.011650 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-a6e59272-157c-4b27-af56-0e57f03089ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:20:52.113135 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1183/2363] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:20:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:20:52.122622 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04afaf6d-f819-4530-8f6f-5039b708b705 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:20:52.122871 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04afaf6d-f819-4530-8f6f-5039b708b705 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:20:52.123159 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04afaf6d-f819-4530-8f6f-5039b708b705 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:20:52.144936 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-04afaf6d-f819-4530-8f6f-5039b708b705 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:20:52.146167 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-04afaf6d-f819-4530-8f6f-5039b708b705 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:20:52.156332 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1181/2364] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:20:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:20:52.161287 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e59272-157c-4b27-af56-0e57f03089ae None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:20:52.174671 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e59272-157c-4b27-af56-0e57f03089ae None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:20:52.180976 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4e576e-e385-44da-8501-9eda8f8df25b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:20:52.181800 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4e576e-e385-44da-8501-9eda8f8df25b admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:20:52.182082 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4e576e-e385-44da-8501-9eda8f8df25b admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:20:52.182452 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4e576e-e385-44da-8501-9eda8f8df25b admin admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:20:52.183103 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4e576e-e385-44da-8501-9eda8f8df25b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:20:52.183418 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4e576e-e385-44da-8501-9eda8f8df25b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:20:52.187234 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4e576e-e385-44da-8501-9eda8f8df25b admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:20:52.211361 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1184/2365] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Jul 25 12:20:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 12:20:52.211495 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04afaf6d-f819-4530-8f6f-5039b708b705 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:20:52.221381 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04afaf6d-f819-4530-8f6f-5039b708b705 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:20:52.226310 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ba18dc-9177-4608-b9bf-28fb3cdc7cb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:20:52.226975 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ba18dc-9177-4608-b9bf-28fb3cdc7cb1 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:20:52.227266 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ba18dc-9177-4608-b9bf-28fb3cdc7cb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:20:52.228135 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ba18dc-9177-4608-b9bf-28fb3cdc7cb1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:20:52.228854 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ba18dc-9177-4608-b9bf-28fb3cdc7cb1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:20:52.230790 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ba18dc-9177-4608-b9bf-28fb3cdc7cb1 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:20:52.260758 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1182/2366] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Jul 25 12:20:52 2022] POST /identity/v3/projects => generated 383 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:20:52.261964 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4e576e-e385-44da-8501-9eda8f8df25b admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:20:52.272230 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4e576e-e385-44da-8501-9eda8f8df25b admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:20:52.334063 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52867d07-6bd7-424e-adbf-a9534bf50268 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:20:52.334731 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52867d07-6bd7-424e-adbf-a9534bf50268 admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:20:52.335021 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52867d07-6bd7-424e-adbf-a9534bf50268 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:20:52.335303 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52867d07-6bd7-424e-adbf-a9534bf50268 admin admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:20:52.336038 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52867d07-6bd7-424e-adbf-a9534bf50268 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:20:52.338031 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52867d07-6bd7-424e-adbf-a9534bf50268 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:20:52.358418 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1185/2367] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Jul 25 12:20:52 2022] POST /identity/v3/users => generated 292 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 12:20:52.362515 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ba18dc-9177-4608-b9bf-28fb3cdc7cb1 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:20:52.373619 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ba18dc-9177-4608-b9bf-28fb3cdc7cb1 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:20:52.378329 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41264298-e421-4eae-a20c-d9e30bf42db9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:20:52.378984 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41264298-e421-4eae-a20c-d9e30bf42db9 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:20:52.379302 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41264298-e421-4eae-a20c-d9e30bf42db9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:20:52.379602 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41264298-e421-4eae-a20c-d9e30bf42db9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:20:52.380330 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41264298-e421-4eae-a20c-d9e30bf42db9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:20:52.381923 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41264298-e421-4eae-a20c-d9e30bf42db9 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:20:52.392293 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1183/2368] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Jul 25 12:20:52 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:20:52.396101 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52867d07-6bd7-424e-adbf-a9534bf50268 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:20:52.407808 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52867d07-6bd7-424e-adbf-a9534bf50268 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:20:52.413475 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68421abf-005f-4e06-ae7a-bbd6134b4853 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:20:52.414179 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68421abf-005f-4e06-ae7a-bbd6134b4853 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:20:52.414511 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68421abf-005f-4e06-ae7a-bbd6134b4853 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:20:52.414833 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68421abf-005f-4e06-ae7a-bbd6134b4853 admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:20:52.415616 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68421abf-005f-4e06-ae7a-bbd6134b4853 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:20:52.417257 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68421abf-005f-4e06-ae7a-bbd6134b4853 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:20:52.427501 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1186/2369] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Jul 25 12:20:52 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:20:52.432353 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41264298-e421-4eae-a20c-d9e30bf42db9 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:20:52.442264 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41264298-e421-4eae-a20c-d9e30bf42db9 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:20:52.447515 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def8db7c-2ce0-44d2-a56d-a8e3b8257a65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:20:52.448412 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def8db7c-2ce0-44d2-a56d-a8e3b8257a65 admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:20:52.448702 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def8db7c-2ce0-44d2-a56d-a8e3b8257a65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:20:52.448986 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def8db7c-2ce0-44d2-a56d-a8e3b8257a65 admin admin] PATH_INFO: `/v3/projects/0ae591381fcc4df29ec9255e3b24e97e/users/4f738e8282bf4a56829743d87cae4cdc/roles/472313ae69174578824df0bed6390744` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:20:52.463475 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def8db7c-2ce0-44d2-a56d-a8e3b8257a65 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ae591381fcc4df29ec9255e3b24e97e, user_id=4f738e8282bf4a56829743d87cae4cdc, role_id=472313ae69174578824df0bed6390744)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:20:52.466159 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def8db7c-2ce0-44d2-a56d-a8e3b8257a65 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:20:52.478856 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1184/2370] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Jul 25 12:20:52 2022] PUT /identity/v3/projects/0ae591381fcc4df29ec9255e3b24e97e/users/4f738e8282bf4a56829743d87cae4cdc/roles/472313ae69174578824df0bed6390744 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:20:52.489047 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a3348f-a7f8-44f7-90e8-5a8781baf3df None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:20:52.489384 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a3348f-a7f8-44f7-90e8-5a8781baf3df None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:20:52.489705 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a3348f-a7f8-44f7-90e8-5a8781baf3df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:20:52.518574 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-91a3348f-a7f8-44f7-90e8-5a8781baf3df None None] MFA Rules not processed for user `4f738e8282bf4a56829743d87cae4cdc`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:20:52.519828 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-91a3348f-a7f8-44f7-90e8-5a8781baf3df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:20:52.561834 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:20:52.585079 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1187/2371] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:20:52 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:20:52.589768 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def8db7c-2ce0-44d2-a56d-a8e3b8257a65 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:20:52.613343 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:20:52.656790 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def8db7c-2ce0-44d2-a56d-a8e3b8257a65 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:20:52.661452 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60389f03-84a7-43f7-a829-85b8a80761f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:20:52.662105 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60389f03-84a7-43f7-a829-85b8a80761f7 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:20:52.662389 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60389f03-84a7-43f7-a829-85b8a80761f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:20:52.662665 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60389f03-84a7-43f7-a829-85b8a80761f7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:20:52.663362 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60389f03-84a7-43f7-a829-85b8a80761f7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:20:52.665787 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60389f03-84a7-43f7-a829-85b8a80761f7 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:20:52.694379 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1185/2372] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Jul 25 12:20:52 2022] POST /identity/v3/projects => generated 383 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 12:20:52.697854 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a3348f-a7f8-44f7-90e8-5a8781baf3df None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:20:52.708623 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a3348f-a7f8-44f7-90e8-5a8781baf3df None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:20:52.764580 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26988bdd-e460-489a-bebd-1240c808ba7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:20:52.765257 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26988bdd-e460-489a-bebd-1240c808ba7d admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:20:52.765551 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26988bdd-e460-489a-bebd-1240c808ba7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:20:52.765836 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26988bdd-e460-489a-bebd-1240c808ba7d admin admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:20:52.766528 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26988bdd-e460-489a-bebd-1240c808ba7d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:20:52.768509 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26988bdd-e460-489a-bebd-1240c808ba7d admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:20:52.790653 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1188/2373] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Jul 25 12:20:52 2022] POST /identity/v3/users => generated 298 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 12:20:52.794727 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60389f03-84a7-43f7-a829-85b8a80761f7 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:20:52.805148 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60389f03-84a7-43f7-a829-85b8a80761f7 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:20:52.810429 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff97defb-5c8e-4571-a8af-e73ab942e068 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:20:52.811113 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff97defb-5c8e-4571-a8af-e73ab942e068 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:20:52.811407 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff97defb-5c8e-4571-a8af-e73ab942e068 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:20:52.811732 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff97defb-5c8e-4571-a8af-e73ab942e068 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:20:52.812466 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff97defb-5c8e-4571-a8af-e73ab942e068 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:20:52.814065 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff97defb-5c8e-4571-a8af-e73ab942e068 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:20:52.824534 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1186/2374] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Jul 25 12:20:52 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:20:52.828515 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26988bdd-e460-489a-bebd-1240c808ba7d admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:20:52.838807 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26988bdd-e460-489a-bebd-1240c808ba7d admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:20:52.843472 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bd8396-0b1f-4154-bfac-50da535638e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:20:52.844412 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bd8396-0b1f-4154-bfac-50da535638e4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:20:52.844703 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bd8396-0b1f-4154-bfac-50da535638e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:20:52.844987 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bd8396-0b1f-4154-bfac-50da535638e4 admin admin] PATH_INFO: `/v3/projects/c5eeec1c32994707a9798130e4c6814e/users/372a565d75de4e689b2c5ee510e48107/roles/a7d20885b6d94928a34b3cc5b3a00006` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:20:52.860137 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bd8396-0b1f-4154-bfac-50da535638e4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c5eeec1c32994707a9798130e4c6814e, user_id=372a565d75de4e689b2c5ee510e48107, role_id=a7d20885b6d94928a34b3cc5b3a00006)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:20:52.862778 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bd8396-0b1f-4154-bfac-50da535638e4 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:20:52.876028 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1189/2375] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Jul 25 12:20:52 2022] PUT /identity/v3/projects/c5eeec1c32994707a9798130e4c6814e/users/372a565d75de4e689b2c5ee510e48107/roles/a7d20885b6d94928a34b3cc5b3a00006 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:20:52.879910 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff97defb-5c8e-4571-a8af-e73ab942e068 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:20:52.889952 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff97defb-5c8e-4571-a8af-e73ab942e068 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:20:52.946911 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f669e47-c829-430d-8ba4-13e8fce3b55a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:20:52.947602 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f669e47-c829-430d-8ba4-13e8fce3b55a admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:20:52.947898 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f669e47-c829-430d-8ba4-13e8fce3b55a admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:20:52.948191 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f669e47-c829-430d-8ba4-13e8fce3b55a admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:20:52.948926 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f669e47-c829-430d-8ba4-13e8fce3b55a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:20:52.950543 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f669e47-c829-430d-8ba4-13e8fce3b55a admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:20:52.963989 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1187/2376] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Jul 25 12:20:52 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:20:52.968407 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bd8396-0b1f-4154-bfac-50da535638e4 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:20:52.979765 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bd8396-0b1f-4154-bfac-50da535638e4 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:20:52.984752 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd040ed-360c-43de-97b4-449e24c747e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:20:52.985829 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd040ed-360c-43de-97b4-449e24c747e3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:20:52.986380 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd040ed-360c-43de-97b4-449e24c747e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:20:52.986380 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd040ed-360c-43de-97b4-449e24c747e3 admin admin] PATH_INFO: `/v3/projects/c5eeec1c32994707a9798130e4c6814e/users/372a565d75de4e689b2c5ee510e48107/roles/a7d20885b6d94928a34b3cc5b3a00006` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:20:52.988597 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd040ed-360c-43de-97b4-449e24c747e3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c5eeec1c32994707a9798130e4c6814e, user_id=372a565d75de4e689b2c5ee510e48107, role_id=a7d20885b6d94928a34b3cc5b3a00006)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:20:52.991216 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd040ed-360c-43de-97b4-449e24c747e3 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:20:53.001902 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1190/2377] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Jul 25 12:20:52 2022] PUT /identity/v3/projects/c5eeec1c32994707a9798130e4c6814e/users/372a565d75de4e689b2c5ee510e48107/roles/a7d20885b6d94928a34b3cc5b3a00006 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 12:20:53.013074 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798e39ce-6c70-41fe-9523-f7aaab6561f2 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:20:53.013372 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798e39ce-6c70-41fe-9523-f7aaab6561f2 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:20:53.013658 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798e39ce-6c70-41fe-9523-f7aaab6561f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:20:53.042970 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-798e39ce-6c70-41fe-9523-f7aaab6561f2 None None] MFA Rules not processed for user `372a565d75de4e689b2c5ee510e48107`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:20:53.044240 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-798e39ce-6c70-41fe-9523-f7aaab6561f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:20:53.111420 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1188/2378] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:20:53 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:20:53.122184 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd040ed-360c-43de-97b4-449e24c747e3 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:20:53.123917 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-3dd040ed-360c-43de-97b4-449e24c747e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:20:53.189009 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd040ed-360c-43de-97b4-449e24c747e3 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:20:53.192699 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484f98d7-05a8-45f4-a521-03daab6ae27a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f89ce6f0cf04373a414d927fa0f8a7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:20:53.193345 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484f98d7-05a8-45f4-a521-03daab6ae27a service cinder] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:20:53.193628 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484f98d7-05a8-45f4-a521-03daab6ae27a service cinder] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:20:53.193904 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484f98d7-05a8-45f4-a521-03daab6ae27a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:20:53.240487 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:20:53.251572 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484f98d7-05a8-45f4-a521-03daab6ae27a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:20:53.253010 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484f98d7-05a8-45f4-a521-03daab6ae27a service cinder] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:20:53.262525 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1191/2379] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:20:53 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:20:55.306129 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798e39ce-6c70-41fe-9523-f7aaab6561f2 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:20:55.307477 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-798e39ce-6c70-41fe-9523-f7aaab6561f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:20:55.330664 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:20:55.417743 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3ab150-aa45-41e5-b436-a66643935336 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:20:55.418270 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3ab150-aa45-41e5-b436-a66643935336 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:20:55.418358 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3ab150-aa45-41e5-b436-a66643935336 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:20:55.427469 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798e39ce-6c70-41fe-9523-f7aaab6561f2 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:20:55.434658 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b06141-8201-4a5e-a191-2b1e6c1b29dd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f67142a1193d47c7bd33d7040268233c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:20:55.435443 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b06141-8201-4a5e-a191-2b1e6c1b29dd service nova] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:20:55.435796 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b06141-8201-4a5e-a191-2b1e6c1b29dd service nova] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:20:55.436087 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b06141-8201-4a5e-a191-2b1e6c1b29dd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:20:55.438829 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-21b06141-8201-4a5e-a191-2b1e6c1b29dd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:20:55.454859 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b06141-8201-4a5e-a191-2b1e6c1b29dd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:20:55.456556 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b06141-8201-4a5e-a191-2b1e6c1b29dd service nova] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:20:55.471764 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1189/2380] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:20:55 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:20:55.480491 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-7f3ab150-aa45-41e5-b436-a66643935336 None None] MFA Rules not processed for user `93207b858d294d049fa56d45810a40aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:20:55.481751 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-7f3ab150-aa45-41e5-b436-a66643935336 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:20:55.566955 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1192/2381] 10.222.0.42 () {58 vars in 967 bytes} [Mon Jul 25 12:20:55 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 12:20:55.585429 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b06141-8201-4a5e-a191-2b1e6c1b29dd service nova] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:20:55.587154 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-21b06141-8201-4a5e-a191-2b1e6c1b29dd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:20:55.669750 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b06141-8201-4a5e-a191-2b1e6c1b29dd service nova] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:20:55.674974 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e29cd6-8bdb-41b9-9f88-bc41fbe9c812 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bd782ea66364eeb96592096eae63d4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:20:55.676261 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e29cd6-8bdb-41b9-9f88-bc41fbe9c812 service swift] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:20:55.676261 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e29cd6-8bdb-41b9-9f88-bc41fbe9c812 service swift] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:20:55.676446 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e29cd6-8bdb-41b9-9f88-bc41fbe9c812 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:20:55.691177 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e29cd6-8bdb-41b9-9f88-bc41fbe9c812 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:20:55.694444 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e29cd6-8bdb-41b9-9f88-bc41fbe9c812 service swift] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:20:55.707358 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1190/2382] 10.222.0.42 () {60 vars in 1384 bytes} [Mon Jul 25 12:20:55 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 25 12:20:57.656815 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3ab150-aa45-41e5-b436-a66643935336 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:20:57.659001 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-7f3ab150-aa45-41e5-b436-a66643935336 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:20:57.671174 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3ab150-aa45-41e5-b436-a66643935336 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:20:57.674855 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c512007-f3f2-4026-9266-02b2b2b80ec8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f89ce6f0cf04373a414d927fa0f8a7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:20:57.675590 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c512007-f3f2-4026-9266-02b2b2b80ec8 service cinder] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:20:57.675885 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c512007-f3f2-4026-9266-02b2b2b80ec8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:20:57.676168 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c512007-f3f2-4026-9266-02b2b2b80ec8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:20:57.689883 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c512007-f3f2-4026-9266-02b2b2b80ec8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:20:57.691252 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c512007-f3f2-4026-9266-02b2b2b80ec8 service cinder] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:20:57.701425 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1193/2383] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:20:57 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:21:01.115573 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e29cd6-8bdb-41b9-9f88-bc41fbe9c812 service swift] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:21:01.116947 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-35e29cd6-8bdb-41b9-9f88-bc41fbe9c812 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:21:01.173735 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e29cd6-8bdb-41b9-9f88-bc41fbe9c812 service swift] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:21:01.177314 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b63390-acfe-48f7-9864-c288c1792144 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51733781201243e8abf9eca39b207750', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:21:01.178027 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b63390-acfe-48f7-9864-c288c1792144 service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:01.178312 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b63390-acfe-48f7-9864-c288c1792144 service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:01.178585 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b63390-acfe-48f7-9864-c288c1792144 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:01.180384 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-c7b63390-acfe-48f7-9864-c288c1792144 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:21:01.224685 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:21:01.244520 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b63390-acfe-48f7-9864-c288c1792144 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:21:01.245978 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b63390-acfe-48f7-9864-c288c1792144 service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:21:01.255747 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1191/2384] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:21:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:21:02.604981 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c512007-f3f2-4026-9266-02b2b2b80ec8 service cinder] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:21:02.618332 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c512007-f3f2-4026-9266-02b2b2b80ec8 service cinder] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:21:02.630538 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1ebb87-285e-48fa-9402-d6546e318bb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:21:02.630538 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1ebb87-285e-48fa-9402-d6546e318bb2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:02.630538 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1ebb87-285e-48fa-9402-d6546e318bb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:02.630538 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1ebb87-285e-48fa-9402-d6546e318bb2 admin admin] PATH_INFO: `/v3/users/4f738e8282bf4a56829743d87cae4cdc` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:02.631028 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1ebb87-285e-48fa-9402-d6546e318bb2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4f738e8282bf4a56829743d87cae4cdc)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:21:02.631028 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1ebb87-285e-48fa-9402-d6546e318bb2 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:21:02.686948 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-2c1ebb87-285e-48fa-9402-d6546e318bb2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4f738e8282bf4a56829743d87cae4cdc'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:02.687713 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-2c1ebb87-285e-48fa-9402-d6546e318bb2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4f738e8282bf4a56829743d87cae4cdc'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:02.699515 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-2c1ebb87-285e-48fa-9402-d6546e318bb2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4f738e8282bf4a56829743d87cae4cdc'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:02.722299 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-2c1ebb87-285e-48fa-9402-d6546e318bb2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4f738e8282bf4a56829743d87cae4cdc'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:02.733366 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-2c1ebb87-285e-48fa-9402-d6546e318bb2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4f738e8282bf4a56829743d87cae4cdc'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:02.744578 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1194/2385] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Jul 25 12:21:02 2022] DELETE /identity/v3/users/4f738e8282bf4a56829743d87cae4cdc => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:21:02.749435 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7459c3-2b0c-434f-a942-ebe9a7449738 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:02.749891 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7459c3-2b0c-434f-a942-ebe9a7449738 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:02.750012 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7459c3-2b0c-434f-a942-ebe9a7449738 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:02.779059 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-1e7459c3-2b0c-434f-a942-ebe9a7449738 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:21:02.780332 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-1e7459c3-2b0c-434f-a942-ebe9a7449738 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:21:02.847062 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1192/2386] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:21:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:21:02.858035 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1ebb87-285e-48fa-9402-d6546e318bb2 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:21:02.859782 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-2c1ebb87-285e-48fa-9402-d6546e318bb2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:21:02.946884 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1ebb87-285e-48fa-9402-d6546e318bb2 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:21:02.950674 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2fac95-f146-4f91-8df6-62a0d0055a8e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51733781201243e8abf9eca39b207750', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:21:02.951394 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2fac95-f146-4f91-8df6-62a0d0055a8e service neutron] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:02.952666 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2fac95-f146-4f91-8df6-62a0d0055a8e service neutron] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:02.952956 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2fac95-f146-4f91-8df6-62a0d0055a8e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:02.965064 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:21:02.968024 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2fac95-f146-4f91-8df6-62a0d0055a8e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:21:02.969678 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2fac95-f146-4f91-8df6-62a0d0055a8e service neutron] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:21:02.984264 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1195/2387] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:21:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:21:03.417219 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7459c3-2b0c-434f-a942-ebe9a7449738 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:21:03.418679 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-1e7459c3-2b0c-434f-a942-ebe9a7449738 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:21:03.432634 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7459c3-2b0c-434f-a942-ebe9a7449738 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:21:03.437322 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d9081b-3bfc-41e8-a94b-0dbfec5f433a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:21:03.438193 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d9081b-3bfc-41e8-a94b-0dbfec5f433a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:03.438477 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d9081b-3bfc-41e8-a94b-0dbfec5f433a admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:03.438759 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d9081b-3bfc-41e8-a94b-0dbfec5f433a admin admin] PATH_INFO: `/v3/projects/0ae591381fcc4df29ec9255e3b24e97e` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:03.439987 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d9081b-3bfc-41e8-a94b-0dbfec5f433a admin admin] RBAC: Authorizing `identity:delete_project(project_id=0ae591381fcc4df29ec9255e3b24e97e)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:21:03.441915 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d9081b-3bfc-41e8-a94b-0dbfec5f433a admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:21:03.520091 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-81d9081b-3bfc-41e8-a94b-0dbfec5f433a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0ae591381fcc4df29ec9255e3b24e97e', 'request_id': 'req-81d9081b-3bfc-41e8-a94b-0dbfec5f433a'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:03.530577 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-81d9081b-3bfc-41e8-a94b-0dbfec5f433a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0ae591381fcc4df29ec9255e3b24e97e', 'request_id': 'req-81d9081b-3bfc-41e8-a94b-0dbfec5f433a'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:03.540365 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-81d9081b-3bfc-41e8-a94b-0dbfec5f433a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0ae591381fcc4df29ec9255e3b24e97e', 'request_id': 'req-81d9081b-3bfc-41e8-a94b-0dbfec5f433a'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:03.555799 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-81d9081b-3bfc-41e8-a94b-0dbfec5f433a admin admin] The token cache is being invalidate because project 0ae591381fcc4df29ec9255e3b24e97e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 12:21:03.556106 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-81d9081b-3bfc-41e8-a94b-0dbfec5f433a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:03.556817 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-81d9081b-3bfc-41e8-a94b-0dbfec5f433a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:03.558802 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1193/2388] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Jul 25 12:21:03 2022] DELETE /identity/v3/projects/0ae591381fcc4df29ec9255e3b24e97e => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:21:03.562486 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2fac95-f146-4f91-8df6-62a0d0055a8e service neutron] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:21:03.574854 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2fac95-f146-4f91-8df6-62a0d0055a8e service neutron] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:21:03.664088 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d5f10c-a437-43f9-b725-bf3adba7ab52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:21:03.665126 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d5f10c-a437-43f9-b725-bf3adba7ab52 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:03.665481 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d5f10c-a437-43f9-b725-bf3adba7ab52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:03.665851 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d5f10c-a437-43f9-b725-bf3adba7ab52 admin admin] PATH_INFO: `/v3/users/372a565d75de4e689b2c5ee510e48107` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:03.667197 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d5f10c-a437-43f9-b725-bf3adba7ab52 admin admin] RBAC: Authorizing `identity:delete_user(user_id=372a565d75de4e689b2c5ee510e48107)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:21:03.669250 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d5f10c-a437-43f9-b725-bf3adba7ab52 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:21:03.720386 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-a7d5f10c-a437-43f9-b725-bf3adba7ab52 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '372a565d75de4e689b2c5ee510e48107'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:03.721109 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-a7d5f10c-a437-43f9-b725-bf3adba7ab52 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '372a565d75de4e689b2c5ee510e48107'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:03.730097 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-a7d5f10c-a437-43f9-b725-bf3adba7ab52 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '372a565d75de4e689b2c5ee510e48107'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:03.742311 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-a7d5f10c-a437-43f9-b725-bf3adba7ab52 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '372a565d75de4e689b2c5ee510e48107'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:03.750202 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-a7d5f10c-a437-43f9-b725-bf3adba7ab52 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '372a565d75de4e689b2c5ee510e48107'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:03.756500 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1196/2389] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Jul 25 12:21:03 2022] DELETE /identity/v3/users/372a565d75de4e689b2c5ee510e48107 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:21:04.058330 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d9081b-3bfc-41e8-a94b-0dbfec5f433a admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:21:04.060058 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-81d9081b-3bfc-41e8-a94b-0dbfec5f433a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:21:04.086226 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d9081b-3bfc-41e8-a94b-0dbfec5f433a admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:21:04.171120 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446fdc9f-1167-4517-95e1-61ebd9362d9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:21:04.171978 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446fdc9f-1167-4517-95e1-61ebd9362d9d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:04.172306 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446fdc9f-1167-4517-95e1-61ebd9362d9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:04.172602 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446fdc9f-1167-4517-95e1-61ebd9362d9d admin admin] PATH_INFO: `/v3/projects/c5eeec1c32994707a9798130e4c6814e` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:04.173886 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446fdc9f-1167-4517-95e1-61ebd9362d9d admin admin] RBAC: Authorizing `identity:delete_project(project_id=c5eeec1c32994707a9798130e4c6814e)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:21:04.176188 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446fdc9f-1167-4517-95e1-61ebd9362d9d admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:21:04.190402 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:21:04.241417 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-446fdc9f-1167-4517-95e1-61ebd9362d9d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c5eeec1c32994707a9798130e4c6814e', 'request_id': 'req-446fdc9f-1167-4517-95e1-61ebd9362d9d'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:04.254866 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-446fdc9f-1167-4517-95e1-61ebd9362d9d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c5eeec1c32994707a9798130e4c6814e', 'request_id': 'req-446fdc9f-1167-4517-95e1-61ebd9362d9d'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:04.266077 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-446fdc9f-1167-4517-95e1-61ebd9362d9d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c5eeec1c32994707a9798130e4c6814e', 'request_id': 'req-446fdc9f-1167-4517-95e1-61ebd9362d9d'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:04.294411 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-446fdc9f-1167-4517-95e1-61ebd9362d9d admin admin] The token cache is being invalidate because project c5eeec1c32994707a9798130e4c6814e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 12:21:04.294692 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-446fdc9f-1167-4517-95e1-61ebd9362d9d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:04.295400 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-446fdc9f-1167-4517-95e1-61ebd9362d9d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:04.297459 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1194/2390] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Jul 25 12:21:04 2022] DELETE /identity/v3/projects/c5eeec1c32994707a9798130e4c6814e => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:21:04.303421 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344bedd0-b21a-4999-81e0-2930ae03a945 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:04.303861 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344bedd0-b21a-4999-81e0-2930ae03a945 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:04.304244 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344bedd0-b21a-4999-81e0-2930ae03a945 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:04.328500 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-344bedd0-b21a-4999-81e0-2930ae03a945 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:21:04.329753 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-344bedd0-b21a-4999-81e0-2930ae03a945 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:21:04.399833 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1197/2391] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:21:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:21:04.411266 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d29f0ac-4429-4f97-bcf7-d68c73c892c0 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:04.411822 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d29f0ac-4429-4f97-bcf7-d68c73c892c0 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:04.412727 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d29f0ac-4429-4f97-bcf7-d68c73c892c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:04.435369 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-5d29f0ac-4429-4f97-bcf7-d68c73c892c0 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:21:04.436966 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-5d29f0ac-4429-4f97-bcf7-d68c73c892c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:21:04.448294 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1195/2392] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:21:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:21:04.453449 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344bedd0-b21a-4999-81e0-2930ae03a945 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:21:04.473105 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344bedd0-b21a-4999-81e0-2930ae03a945 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:21:04.479205 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed22b18-192f-4736-81f3-b397651b9991 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:21:04.480013 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed22b18-192f-4736-81f3-b397651b9991 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:04.480305 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed22b18-192f-4736-81f3-b397651b9991 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:04.480587 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed22b18-192f-4736-81f3-b397651b9991 admin admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:04.482101 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed22b18-192f-4736-81f3-b397651b9991 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:21:04.482772 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed22b18-192f-4736-81f3-b397651b9991 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:21:04.484252 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed22b18-192f-4736-81f3-b397651b9991 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:21:04.501149 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1198/2393] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Jul 25 12:21:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:21:04.501659 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d29f0ac-4429-4f97-bcf7-d68c73c892c0 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:21:04.512827 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d29f0ac-4429-4f97-bcf7-d68c73c892c0 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:21:04.517829 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b484ffad-205e-4c16-88ee-726e4cedf76e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:21:04.518505 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b484ffad-205e-4c16-88ee-726e4cedf76e admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:04.518794 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b484ffad-205e-4c16-88ee-726e4cedf76e admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:04.519165 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b484ffad-205e-4c16-88ee-726e4cedf76e admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:04.520339 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b484ffad-205e-4c16-88ee-726e4cedf76e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:21:04.521966 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b484ffad-205e-4c16-88ee-726e4cedf76e admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:21:04.559580 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1196/2394] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Jul 25 12:21:04 2022] POST /identity/v3/projects => generated 369 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 12:21:04.561339 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed22b18-192f-4736-81f3-b397651b9991 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:21:04.567126 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:21:04.574038 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed22b18-192f-4736-81f3-b397651b9991 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:21:04.655536 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b077622-809a-47e3-9c23-ba858cd78481 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:21:04.656335 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b077622-809a-47e3-9c23-ba858cd78481 admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:04.656631 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b077622-809a-47e3-9c23-ba858cd78481 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:04.657037 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b077622-809a-47e3-9c23-ba858cd78481 admin admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:04.658379 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b077622-809a-47e3-9c23-ba858cd78481 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:21:04.660402 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b077622-809a-47e3-9c23-ba858cd78481 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:21:04.693013 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1199/2395] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Jul 25 12:21:04 2022] POST /identity/v3/users => generated 285 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:21:04.697582 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b484ffad-205e-4c16-88ee-726e4cedf76e admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:21:04.715790 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b484ffad-205e-4c16-88ee-726e4cedf76e admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:21:04.728847 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aef23b8-0419-4741-b4a7-102d856ad044 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:21:04.730372 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aef23b8-0419-4741-b4a7-102d856ad044 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:04.731336 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aef23b8-0419-4741-b4a7-102d856ad044 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:04.733240 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aef23b8-0419-4741-b4a7-102d856ad044 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:04.734791 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aef23b8-0419-4741-b4a7-102d856ad044 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:21:04.740303 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aef23b8-0419-4741-b4a7-102d856ad044 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:21:04.772140 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1197/2396] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Jul 25 12:21:04 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:21:04.773380 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b077622-809a-47e3-9c23-ba858cd78481 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:21:04.787905 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b077622-809a-47e3-9c23-ba858cd78481 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:21:04.793652 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f56647a-a4c5-4cfd-8e7f-e5b2ef8bde03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:21:04.794348 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f56647a-a4c5-4cfd-8e7f-e5b2ef8bde03 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:04.794718 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f56647a-a4c5-4cfd-8e7f-e5b2ef8bde03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:04.795007 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f56647a-a4c5-4cfd-8e7f-e5b2ef8bde03 admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:04.798254 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f56647a-a4c5-4cfd-8e7f-e5b2ef8bde03 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:21:04.807109 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f56647a-a4c5-4cfd-8e7f-e5b2ef8bde03 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:21:04.838971 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1200/2397] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Jul 25 12:21:04 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 12:21:04.840442 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aef23b8-0419-4741-b4a7-102d856ad044 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:21:04.855262 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aef23b8-0419-4741-b4a7-102d856ad044 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:21:04.860376 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e99db5-58f1-4b54-b1f8-42e52e2e5e4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:21:04.861469 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e99db5-58f1-4b54-b1f8-42e52e2e5e4c admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:04.861819 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e99db5-58f1-4b54-b1f8-42e52e2e5e4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:04.862107 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e99db5-58f1-4b54-b1f8-42e52e2e5e4c admin admin] PATH_INFO: `/v3/projects/d9103a1dee084ce49690cbb3f72ad65b/users/5818e45e4ba4403e9a5ea240fb5d229a/roles/472313ae69174578824df0bed6390744` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:04.887269 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e99db5-58f1-4b54-b1f8-42e52e2e5e4c admin admin] RBAC: Authorizing `identity:create_grant(project_id=d9103a1dee084ce49690cbb3f72ad65b, user_id=5818e45e4ba4403e9a5ea240fb5d229a, role_id=472313ae69174578824df0bed6390744)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:21:04.890116 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e99db5-58f1-4b54-b1f8-42e52e2e5e4c admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:21:04.908709 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1198/2398] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Jul 25 12:21:04 2022] PUT /identity/v3/projects/d9103a1dee084ce49690cbb3f72ad65b/users/5818e45e4ba4403e9a5ea240fb5d229a/roles/472313ae69174578824df0bed6390744 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:21:04.920215 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2794d449-40e4-46c1-abb7-e7f2ef4c0ef9 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:04.920601 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2794d449-40e4-46c1-abb7-e7f2ef4c0ef9 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:04.920952 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2794d449-40e4-46c1-abb7-e7f2ef4c0ef9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:04.947915 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e99db5-58f1-4b54-b1f8-42e52e2e5e4c admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:21:04.951011 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-80e99db5-58f1-4b54-b1f8-42e52e2e5e4c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:21:04.953908 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-2794d449-40e4-46c1-abb7-e7f2ef4c0ef9 None None] MFA Rules not processed for user `5818e45e4ba4403e9a5ea240fb5d229a`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:21:04.956144 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-2794d449-40e4-46c1-abb7-e7f2ef4c0ef9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:21:05.020349 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e99db5-58f1-4b54-b1f8-42e52e2e5e4c admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:21:05.025869 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11617015-c5f7-4390-ba91-9b47d418c403 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:21:05.026771 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11617015-c5f7-4390-ba91-9b47d418c403 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:05.027066 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11617015-c5f7-4390-ba91-9b47d418c403 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:05.027360 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11617015-c5f7-4390-ba91-9b47d418c403 admin admin] PATH_INFO: `/v3/users/c5b98bbcc2d640de837d141f83dfe924` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:05.028608 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11617015-c5f7-4390-ba91-9b47d418c403 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c5b98bbcc2d640de837d141f83dfe924)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:21:05.030526 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11617015-c5f7-4390-ba91-9b47d418c403 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:21:05.032797 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1201/2399] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:21:04 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:21:05.045363 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2794d449-40e4-46c1-abb7-e7f2ef4c0ef9 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:21:05.046798 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-2794d449-40e4-46c1-abb7-e7f2ef4c0ef9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:21:05.066543 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:21:05.083376 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-11617015-c5f7-4390-ba91-9b47d418c403 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c5b98bbcc2d640de837d141f83dfe924'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:05.091252 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-11617015-c5f7-4390-ba91-9b47d418c403 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c5b98bbcc2d640de837d141f83dfe924'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:05.099047 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-11617015-c5f7-4390-ba91-9b47d418c403 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c5b98bbcc2d640de837d141f83dfe924'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:05.107689 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2794d449-40e4-46c1-abb7-e7f2ef4c0ef9 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:21:05.111379 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad439fb0-8e8d-49a2-9000-4e350ec14248 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ea16528b1374bae9cf54603f7907bfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:21:05.111726 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-11617015-c5f7-4390-ba91-9b47d418c403 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c5b98bbcc2d640de837d141f83dfe924'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:05.112499 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-11617015-c5f7-4390-ba91-9b47d418c403 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c5b98bbcc2d640de837d141f83dfe924'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:05.112672 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad439fb0-8e8d-49a2-9000-4e350ec14248 service glance] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:05.113075 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad439fb0-8e8d-49a2-9000-4e350ec14248 service glance] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:05.113362 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad439fb0-8e8d-49a2-9000-4e350ec14248 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:05.115161 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-ad439fb0-8e8d-49a2-9000-4e350ec14248 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:21:05.119240 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1199/2400] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Jul 25 12:21:04 2022] DELETE /identity/v3/users/c5b98bbcc2d640de837d141f83dfe924 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:21:05.126943 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad439fb0-8e8d-49a2-9000-4e350ec14248 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:21:05.128407 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad439fb0-8e8d-49a2-9000-4e350ec14248 service glance] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:21:05.138824 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1202/2401] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:21:05 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 12:21:05.312491 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11617015-c5f7-4390-ba91-9b47d418c403 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:21:05.314210 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-11617015-c5f7-4390-ba91-9b47d418c403 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:21:05.393347 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11617015-c5f7-4390-ba91-9b47d418c403 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:21:05.398146 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abaea0e8-7152-402e-bec8-a8cd357d1a44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:21:05.399010 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abaea0e8-7152-402e-bec8-a8cd357d1a44 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:05.399299 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abaea0e8-7152-402e-bec8-a8cd357d1a44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:05.399621 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abaea0e8-7152-402e-bec8-a8cd357d1a44 admin admin] PATH_INFO: `/v3/projects/ce78c16fa24c475b945a64a7995c70e4` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:05.401159 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abaea0e8-7152-402e-bec8-a8cd357d1a44 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ce78c16fa24c475b945a64a7995c70e4)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:21:05.403078 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abaea0e8-7152-402e-bec8-a8cd357d1a44 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:21:05.454713 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-abaea0e8-7152-402e-bec8-a8cd357d1a44 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ce78c16fa24c475b945a64a7995c70e4', 'request_id': 'req-abaea0e8-7152-402e-bec8-a8cd357d1a44'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:05.462953 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-abaea0e8-7152-402e-bec8-a8cd357d1a44 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ce78c16fa24c475b945a64a7995c70e4', 'request_id': 'req-abaea0e8-7152-402e-bec8-a8cd357d1a44'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:05.470099 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-abaea0e8-7152-402e-bec8-a8cd357d1a44 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ce78c16fa24c475b945a64a7995c70e4', 'request_id': 'req-abaea0e8-7152-402e-bec8-a8cd357d1a44'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:05.484884 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-abaea0e8-7152-402e-bec8-a8cd357d1a44 admin admin] The token cache is being invalidate because project ce78c16fa24c475b945a64a7995c70e4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 12:21:05.485186 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-abaea0e8-7152-402e-bec8-a8cd357d1a44 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:05.485856 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-abaea0e8-7152-402e-bec8-a8cd357d1a44 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:05.487419 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1200/2402] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Jul 25 12:21:05 2022] DELETE /identity/v3/projects/ce78c16fa24c475b945a64a7995c70e4 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:21:05.493228 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5018c7fe-9cd0-4df3-9dce-39a4a5733e43 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:05.493621 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5018c7fe-9cd0-4df3-9dce-39a4a5733e43 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:05.493907 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5018c7fe-9cd0-4df3-9dce-39a4a5733e43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:05.513300 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-5018c7fe-9cd0-4df3-9dce-39a4a5733e43 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:21:05.514568 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-5018c7fe-9cd0-4df3-9dce-39a4a5733e43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:21:05.525623 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:21:05.594919 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1203/2403] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:21:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:21:05.596816 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8db5744-ebd8-40fb-ad74-61257f32e8e2 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:05.597180 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8db5744-ebd8-40fb-ad74-61257f32e8e2 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:05.597535 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8db5744-ebd8-40fb-ad74-61257f32e8e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:05.628072 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-b8db5744-ebd8-40fb-ad74-61257f32e8e2 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:21:05.629615 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-b8db5744-ebd8-40fb-ad74-61257f32e8e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:21:05.641090 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1201/2404] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:21:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:21:05.645212 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5018c7fe-9cd0-4df3-9dce-39a4a5733e43 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:21:05.656430 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5018c7fe-9cd0-4df3-9dce-39a4a5733e43 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:21:05.662022 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777bd04e-f89a-4301-af26-e7d055747de5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:21:05.662699 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777bd04e-f89a-4301-af26-e7d055747de5 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:05.662993 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777bd04e-f89a-4301-af26-e7d055747de5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:05.663276 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777bd04e-f89a-4301-af26-e7d055747de5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:05.664915 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777bd04e-f89a-4301-af26-e7d055747de5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:21:05.665429 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777bd04e-f89a-4301-af26-e7d055747de5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:21:05.667128 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777bd04e-f89a-4301-af26-e7d055747de5 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:21:05.683987 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1204/2405] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Jul 25 12:21:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:21:05.685301 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8db5744-ebd8-40fb-ad74-61257f32e8e2 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:21:05.695983 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8db5744-ebd8-40fb-ad74-61257f32e8e2 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:21:05.700882 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef38074d-8819-47dc-a677-a8afeb316dbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:21:05.701545 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef38074d-8819-47dc-a677-a8afeb316dbd admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:05.701839 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef38074d-8819-47dc-a677-a8afeb316dbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:05.702126 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef38074d-8819-47dc-a677-a8afeb316dbd admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:05.702824 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef38074d-8819-47dc-a677-a8afeb316dbd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:21:05.704842 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef38074d-8819-47dc-a677-a8afeb316dbd admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:21:05.733965 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1202/2406] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Jul 25 12:21:05 2022] POST /identity/v3/projects => generated 383 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:21:05.738174 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777bd04e-f89a-4301-af26-e7d055747de5 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:21:05.748517 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777bd04e-f89a-4301-af26-e7d055747de5 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:21:05.822886 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d969dc-a571-45ae-af51-07879c1a4c4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:21:05.823677 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d969dc-a571-45ae-af51-07879c1a4c4f admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:05.823970 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d969dc-a571-45ae-af51-07879c1a4c4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:05.824262 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d969dc-a571-45ae-af51-07879c1a4c4f admin admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:05.824976 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d969dc-a571-45ae-af51-07879c1a4c4f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:21:05.826945 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d969dc-a571-45ae-af51-07879c1a4c4f admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:21:05.850444 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1205/2407] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Jul 25 12:21:05 2022] POST /identity/v3/users => generated 292 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:21:05.860609 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef38074d-8819-47dc-a677-a8afeb316dbd admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:21:05.871243 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef38074d-8819-47dc-a677-a8afeb316dbd admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:21:05.875964 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7513595-c649-445f-a77e-a4c40951d9db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:21:05.876631 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7513595-c649-445f-a77e-a4c40951d9db admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:05.876921 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7513595-c649-445f-a77e-a4c40951d9db admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:05.877204 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7513595-c649-445f-a77e-a4c40951d9db admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:05.877968 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7513595-c649-445f-a77e-a4c40951d9db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:21:05.879769 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7513595-c649-445f-a77e-a4c40951d9db admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:21:05.885389 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:21:05.890870 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1203/2408] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Jul 25 12:21:05 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:21:05.895116 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d969dc-a571-45ae-af51-07879c1a4c4f admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:21:05.908210 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d969dc-a571-45ae-af51-07879c1a4c4f admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:21:05.913408 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba6ee72-ac3c-4c92-ad14-8ec012d641a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:21:05.914087 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba6ee72-ac3c-4c92-ad14-8ec012d641a0 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:05.915111 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba6ee72-ac3c-4c92-ad14-8ec012d641a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:05.915812 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba6ee72-ac3c-4c92-ad14-8ec012d641a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:05.916365 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba6ee72-ac3c-4c92-ad14-8ec012d641a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:21:05.918058 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba6ee72-ac3c-4c92-ad14-8ec012d641a0 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:21:05.930549 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1206/2409] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Jul 25 12:21:05 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:21:05.934737 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7513595-c649-445f-a77e-a4c40951d9db admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:21:05.945235 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7513595-c649-445f-a77e-a4c40951d9db admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:21:05.950202 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75dbde9b-a043-4c53-9817-579f48b6003a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:21:05.951089 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dbde9b-a043-4c53-9817-579f48b6003a admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:05.951380 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dbde9b-a043-4c53-9817-579f48b6003a admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:05.951703 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dbde9b-a043-4c53-9817-579f48b6003a admin admin] PATH_INFO: `/v3/projects/8e8c095719274135b37bd1dd4ab156e0/users/ea373cc0bce64e3e970236db62bfcfda/roles/472313ae69174578824df0bed6390744` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:05.968329 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75dbde9b-a043-4c53-9817-579f48b6003a admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e8c095719274135b37bd1dd4ab156e0, user_id=ea373cc0bce64e3e970236db62bfcfda, role_id=472313ae69174578824df0bed6390744)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:21:05.970946 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75dbde9b-a043-4c53-9817-579f48b6003a admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:21:05.988155 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1204/2410] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Jul 25 12:21:05 2022] PUT /identity/v3/projects/8e8c095719274135b37bd1dd4ab156e0/users/ea373cc0bce64e3e970236db62bfcfda/roles/472313ae69174578824df0bed6390744 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 12:21:06.001071 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467e85de-6cf0-4697-9e4c-3eb0b836ee0a None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:06.001376 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467e85de-6cf0-4697-9e4c-3eb0b836ee0a None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:06.001666 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467e85de-6cf0-4697-9e4c-3eb0b836ee0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:06.033794 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-467e85de-6cf0-4697-9e4c-3eb0b836ee0a None None] MFA Rules not processed for user `ea373cc0bce64e3e970236db62bfcfda`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:21:06.035027 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-467e85de-6cf0-4697-9e4c-3eb0b836ee0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:21:06.109559 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1207/2411] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:21:05 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:21:06.114233 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75dbde9b-a043-4c53-9817-579f48b6003a admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:21:06.183679 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75dbde9b-a043-4c53-9817-579f48b6003a admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:21:06.189120 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c096e2-2f6a-418c-9df8-6126b692eb57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:21:06.189794 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c096e2-2f6a-418c-9df8-6126b692eb57 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:06.190092 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c096e2-2f6a-418c-9df8-6126b692eb57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:06.190383 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c096e2-2f6a-418c-9df8-6126b692eb57 admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:06.191106 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c096e2-2f6a-418c-9df8-6126b692eb57 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:21:06.193075 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c096e2-2f6a-418c-9df8-6126b692eb57 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:21:06.225316 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1205/2412] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Jul 25 12:21:06 2022] POST /identity/v3/projects => generated 383 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 12:21:06.226397 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467e85de-6cf0-4697-9e4c-3eb0b836ee0a None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:21:06.237340 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467e85de-6cf0-4697-9e4c-3eb0b836ee0a None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:21:06.242523 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:21:06.301119 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85c21b1-cf63-4aa8-86a3-203a1cba57a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:21:06.301803 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85c21b1-cf63-4aa8-86a3-203a1cba57a0 admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:06.302096 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85c21b1-cf63-4aa8-86a3-203a1cba57a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:06.302393 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85c21b1-cf63-4aa8-86a3-203a1cba57a0 admin admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:06.303078 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85c21b1-cf63-4aa8-86a3-203a1cba57a0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:21:06.305088 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85c21b1-cf63-4aa8-86a3-203a1cba57a0 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:21:06.334428 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1208/2413] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Jul 25 12:21:06 2022] POST /identity/v3/users => generated 298 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:21:06.339277 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c096e2-2f6a-418c-9df8-6126b692eb57 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:21:06.349986 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c096e2-2f6a-418c-9df8-6126b692eb57 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:21:06.354681 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42c2df6-42a7-4430-9f45-7b77f142514c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:21:06.355348 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42c2df6-42a7-4430-9f45-7b77f142514c admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:06.355667 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42c2df6-42a7-4430-9f45-7b77f142514c admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:06.355957 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42c2df6-42a7-4430-9f45-7b77f142514c admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:06.356723 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42c2df6-42a7-4430-9f45-7b77f142514c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:21:06.358327 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42c2df6-42a7-4430-9f45-7b77f142514c admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:21:06.369670 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1206/2414] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Jul 25 12:21:06 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:21:06.374025 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85c21b1-cf63-4aa8-86a3-203a1cba57a0 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:21:06.391203 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85c21b1-cf63-4aa8-86a3-203a1cba57a0 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:21:06.396141 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059cff3e-f5e1-491a-8ef9-db8af2ccd074 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:21:06.397062 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059cff3e-f5e1-491a-8ef9-db8af2ccd074 admin admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:06.397352 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059cff3e-f5e1-491a-8ef9-db8af2ccd074 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:06.397750 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059cff3e-f5e1-491a-8ef9-db8af2ccd074 admin admin] PATH_INFO: `/v3/projects/e4796eb37193408782bfef8f30d20017/users/aee4b31fa58d44d98499cf7038be2a70/roles/a7d20885b6d94928a34b3cc5b3a00006` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:06.416478 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059cff3e-f5e1-491a-8ef9-db8af2ccd074 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e4796eb37193408782bfef8f30d20017, user_id=aee4b31fa58d44d98499cf7038be2a70, role_id=a7d20885b6d94928a34b3cc5b3a00006)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:21:06.419183 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059cff3e-f5e1-491a-8ef9-db8af2ccd074 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:21:06.438222 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1209/2415] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Jul 25 12:21:06 2022] PUT /identity/v3/projects/e4796eb37193408782bfef8f30d20017/users/aee4b31fa58d44d98499cf7038be2a70/roles/a7d20885b6d94928a34b3cc5b3a00006 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:21:06.442171 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42c2df6-42a7-4430-9f45-7b77f142514c admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:21:06.453745 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42c2df6-42a7-4430-9f45-7b77f142514c admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:21:06.524248 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0de772a-96b0-4c63-90cc-6be70d1d50df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:21:06.525329 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0de772a-96b0-4c63-90cc-6be70d1d50df admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:06.525621 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0de772a-96b0-4c63-90cc-6be70d1d50df admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:06.525939 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0de772a-96b0-4c63-90cc-6be70d1d50df admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:06.526666 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0de772a-96b0-4c63-90cc-6be70d1d50df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:21:06.528306 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0de772a-96b0-4c63-90cc-6be70d1d50df admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:21:06.541881 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1207/2416] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Jul 25 12:21:06 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:21:06.543011 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059cff3e-f5e1-491a-8ef9-db8af2ccd074 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:21:06.554061 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059cff3e-f5e1-491a-8ef9-db8af2ccd074 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:21:06.559713 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94e4c37-3144-42b2-8b6a-f070f140b319 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:21:06.560690 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94e4c37-3144-42b2-8b6a-f070f140b319 admin admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:06.561047 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94e4c37-3144-42b2-8b6a-f070f140b319 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:06.561396 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94e4c37-3144-42b2-8b6a-f070f140b319 admin admin] PATH_INFO: `/v3/projects/e4796eb37193408782bfef8f30d20017/users/aee4b31fa58d44d98499cf7038be2a70/roles/a7d20885b6d94928a34b3cc5b3a00006` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:06.563494 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94e4c37-3144-42b2-8b6a-f070f140b319 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e4796eb37193408782bfef8f30d20017, user_id=aee4b31fa58d44d98499cf7038be2a70, role_id=a7d20885b6d94928a34b3cc5b3a00006)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:21:06.566265 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94e4c37-3144-42b2-8b6a-f070f140b319 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:21:06.580390 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1210/2417] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Jul 25 12:21:06 2022] PUT /identity/v3/projects/e4796eb37193408782bfef8f30d20017/users/aee4b31fa58d44d98499cf7038be2a70/roles/a7d20885b6d94928a34b3cc5b3a00006 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 12:21:06.590973 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829bef0c-9f53-49df-9c2c-3bf72c6c99f3 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:06.591265 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829bef0c-9f53-49df-9c2c-3bf72c6c99f3 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:06.591580 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829bef0c-9f53-49df-9c2c-3bf72c6c99f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:06.625390 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-829bef0c-9f53-49df-9c2c-3bf72c6c99f3 None None] MFA Rules not processed for user `aee4b31fa58d44d98499cf7038be2a70`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:21:06.626783 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-829bef0c-9f53-49df-9c2c-3bf72c6c99f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:21:06.672515 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:21:06.716571 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1208/2418] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:21:06 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:21:06.727434 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94e4c37-3144-42b2-8b6a-f070f140b319 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:21:06.729194 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-d94e4c37-3144-42b2-8b6a-f070f140b319 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:21:06.811134 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94e4c37-3144-42b2-8b6a-f070f140b319 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:21:06.840676 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880f147f-adc1-4719-b717-032a3f272c48 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f89ce6f0cf04373a414d927fa0f8a7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:21:06.841398 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880f147f-adc1-4719-b717-032a3f272c48 service cinder] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:06.842813 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880f147f-adc1-4719-b717-032a3f272c48 service cinder] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:06.843222 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880f147f-adc1-4719-b717-032a3f272c48 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:06.889663 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880f147f-adc1-4719-b717-032a3f272c48 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:21:06.891381 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880f147f-adc1-4719-b717-032a3f272c48 service cinder] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:21:06.917206 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1211/2419] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:21:06 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:21:07.056526 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829bef0c-9f53-49df-9c2c-3bf72c6c99f3 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:21:07.067177 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829bef0c-9f53-49df-9c2c-3bf72c6c99f3 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:21:07.071953 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d69366-6314-43df-9808-3fe8b4174cb0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f89ce6f0cf04373a414d927fa0f8a7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:21:07.072735 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d69366-6314-43df-9808-3fe8b4174cb0 service cinder] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:07.073056 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d69366-6314-43df-9808-3fe8b4174cb0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:07.076988 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d69366-6314-43df-9808-3fe8b4174cb0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:07.146309 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d69366-6314-43df-9808-3fe8b4174cb0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:21:07.148062 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d69366-6314-43df-9808-3fe8b4174cb0 service cinder] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:21:07.158896 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1209/2420] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:21:07 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:21:09.815772 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880f147f-adc1-4719-b717-032a3f272c48 service cinder] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:21:09.911356 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880f147f-adc1-4719-b717-032a3f272c48 service cinder] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:21:09.917224 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a56220b-5bef-4c24-8218-b4a3f731c391 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:21:09.918194 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a56220b-5bef-4c24-8218-b4a3f731c391 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:09.918382 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a56220b-5bef-4c24-8218-b4a3f731c391 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:09.918668 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a56220b-5bef-4c24-8218-b4a3f731c391 admin admin] PATH_INFO: `/v3/users/ea373cc0bce64e3e970236db62bfcfda` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:09.919916 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a56220b-5bef-4c24-8218-b4a3f731c391 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ea373cc0bce64e3e970236db62bfcfda)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:21:09.921835 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a56220b-5bef-4c24-8218-b4a3f731c391 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:21:09.926600 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:21:09.982402 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-3a56220b-5bef-4c24-8218-b4a3f731c391 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ea373cc0bce64e3e970236db62bfcfda'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:09.983181 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-3a56220b-5bef-4c24-8218-b4a3f731c391 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ea373cc0bce64e3e970236db62bfcfda'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:09.992548 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-3a56220b-5bef-4c24-8218-b4a3f731c391 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ea373cc0bce64e3e970236db62bfcfda'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:10.007571 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-3a56220b-5bef-4c24-8218-b4a3f731c391 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ea373cc0bce64e3e970236db62bfcfda'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:10.015879 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-3a56220b-5bef-4c24-8218-b4a3f731c391 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ea373cc0bce64e3e970236db62bfcfda'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:10.022929 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1212/2421] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Jul 25 12:21:09 2022] DELETE /identity/v3/users/ea373cc0bce64e3e970236db62bfcfda => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:21:10.028398 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40510488-95fb-42ba-abad-8c904daad743 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:10.028903 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40510488-95fb-42ba-abad-8c904daad743 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:10.029002 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40510488-95fb-42ba-abad-8c904daad743 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:10.049007 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-40510488-95fb-42ba-abad-8c904daad743 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:21:10.050305 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-40510488-95fb-42ba-abad-8c904daad743 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:21:10.109800 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1210/2422] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:21:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:21:10.120669 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a56220b-5bef-4c24-8218-b4a3f731c391 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:21:10.122127 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-3a56220b-5bef-4c24-8218-b4a3f731c391 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:21:10.181852 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a56220b-5bef-4c24-8218-b4a3f731c391 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:21:10.185932 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0effcc-06c1-4881-9a5f-9115e4ccfa15 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51733781201243e8abf9eca39b207750', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:21:10.186589 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0effcc-06c1-4881-9a5f-9115e4ccfa15 service neutron] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:10.186884 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0effcc-06c1-4881-9a5f-9115e4ccfa15 service neutron] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:10.187170 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0effcc-06c1-4881-9a5f-9115e4ccfa15 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:10.196377 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0effcc-06c1-4881-9a5f-9115e4ccfa15 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:21:10.197818 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0effcc-06c1-4881-9a5f-9115e4ccfa15 service neutron] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:21:10.209442 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1213/2423] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:21:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:21:10.483932 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40510488-95fb-42ba-abad-8c904daad743 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:21:10.485378 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-40510488-95fb-42ba-abad-8c904daad743 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:21:10.498265 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40510488-95fb-42ba-abad-8c904daad743 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:21:10.502888 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb9a986-27e5-43e3-9c3f-463dfac8e059 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:21:10.503841 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb9a986-27e5-43e3-9c3f-463dfac8e059 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:10.504138 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb9a986-27e5-43e3-9c3f-463dfac8e059 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:10.504425 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb9a986-27e5-43e3-9c3f-463dfac8e059 admin admin] PATH_INFO: `/v3/projects/8e8c095719274135b37bd1dd4ab156e0` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:10.505751 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb9a986-27e5-43e3-9c3f-463dfac8e059 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8e8c095719274135b37bd1dd4ab156e0)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:21:10.507715 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb9a986-27e5-43e3-9c3f-463dfac8e059 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:21:10.561868 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-3eb9a986-27e5-43e3-9c3f-463dfac8e059 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8e8c095719274135b37bd1dd4ab156e0', 'request_id': 'req-3eb9a986-27e5-43e3-9c3f-463dfac8e059'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:10.569577 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-3eb9a986-27e5-43e3-9c3f-463dfac8e059 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8e8c095719274135b37bd1dd4ab156e0', 'request_id': 'req-3eb9a986-27e5-43e3-9c3f-463dfac8e059'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:10.575851 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-3eb9a986-27e5-43e3-9c3f-463dfac8e059 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8e8c095719274135b37bd1dd4ab156e0', 'request_id': 'req-3eb9a986-27e5-43e3-9c3f-463dfac8e059'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:10.576986 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:21:10.589964 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-3eb9a986-27e5-43e3-9c3f-463dfac8e059 admin admin] The token cache is being invalidate because project 8e8c095719274135b37bd1dd4ab156e0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 12:21:10.590287 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-3eb9a986-27e5-43e3-9c3f-463dfac8e059 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:10.590941 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-3eb9a986-27e5-43e3-9c3f-463dfac8e059 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:10.592521 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1211/2424] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Jul 25 12:21:10 2022] DELETE /identity/v3/projects/8e8c095719274135b37bd1dd4ab156e0 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:21:10.596549 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0effcc-06c1-4881-9a5f-9115e4ccfa15 service neutron] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:21:10.607403 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0effcc-06c1-4881-9a5f-9115e4ccfa15 service neutron] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:21:10.672000 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1f739c-c774-4a5e-b87e-4199b0c07784 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:21:10.672890 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1f739c-c774-4a5e-b87e-4199b0c07784 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:10.673180 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1f739c-c774-4a5e-b87e-4199b0c07784 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:10.673469 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1f739c-c774-4a5e-b87e-4199b0c07784 admin admin] PATH_INFO: `/v3/users/aee4b31fa58d44d98499cf7038be2a70` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:10.674670 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1f739c-c774-4a5e-b87e-4199b0c07784 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aee4b31fa58d44d98499cf7038be2a70)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:21:10.677017 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1f739c-c774-4a5e-b87e-4199b0c07784 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:21:10.706735 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:21:10.725001 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-dd1f739c-c774-4a5e-b87e-4199b0c07784 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aee4b31fa58d44d98499cf7038be2a70'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:10.725657 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-dd1f739c-c774-4a5e-b87e-4199b0c07784 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aee4b31fa58d44d98499cf7038be2a70'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:10.732921 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-dd1f739c-c774-4a5e-b87e-4199b0c07784 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aee4b31fa58d44d98499cf7038be2a70'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:10.745507 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-dd1f739c-c774-4a5e-b87e-4199b0c07784 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aee4b31fa58d44d98499cf7038be2a70'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:10.752267 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-dd1f739c-c774-4a5e-b87e-4199b0c07784 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aee4b31fa58d44d98499cf7038be2a70'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:10.761444 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1214/2425] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Jul 25 12:21:10 2022] DELETE /identity/v3/users/aee4b31fa58d44d98499cf7038be2a70 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:21:11.056880 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb9a986-27e5-43e3-9c3f-463dfac8e059 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:21:11.058829 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-3eb9a986-27e5-43e3-9c3f-463dfac8e059 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:21:11.073336 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb9a986-27e5-43e3-9c3f-463dfac8e059 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:21:11.156478 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc735e2-26aa-4aac-ac45-ba812f080437 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:21:11.157367 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc735e2-26aa-4aac-ac45-ba812f080437 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:11.157659 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc735e2-26aa-4aac-ac45-ba812f080437 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:11.158097 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc735e2-26aa-4aac-ac45-ba812f080437 admin admin] PATH_INFO: `/v3/projects/e4796eb37193408782bfef8f30d20017` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:11.159320 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc735e2-26aa-4aac-ac45-ba812f080437 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e4796eb37193408782bfef8f30d20017)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:21:11.167856 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc735e2-26aa-4aac-ac45-ba812f080437 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:21:11.217789 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-edc735e2-26aa-4aac-ac45-ba812f080437 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e4796eb37193408782bfef8f30d20017', 'request_id': 'req-edc735e2-26aa-4aac-ac45-ba812f080437'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:11.226344 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-edc735e2-26aa-4aac-ac45-ba812f080437 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e4796eb37193408782bfef8f30d20017', 'request_id': 'req-edc735e2-26aa-4aac-ac45-ba812f080437'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:11.234816 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-edc735e2-26aa-4aac-ac45-ba812f080437 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e4796eb37193408782bfef8f30d20017', 'request_id': 'req-edc735e2-26aa-4aac-ac45-ba812f080437'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:11.249131 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-edc735e2-26aa-4aac-ac45-ba812f080437 admin admin] The token cache is being invalidate because project e4796eb37193408782bfef8f30d20017 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 12:21:11.249431 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-edc735e2-26aa-4aac-ac45-ba812f080437 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:11.250126 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-edc735e2-26aa-4aac-ac45-ba812f080437 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:11.251709 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1212/2426] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Jul 25 12:21:11 2022] DELETE /identity/v3/projects/e4796eb37193408782bfef8f30d20017 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:21:11.256885 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468a54d4-2aea-42ae-955e-a209bde90286 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:11.257416 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468a54d4-2aea-42ae-955e-a209bde90286 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:11.257535 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468a54d4-2aea-42ae-955e-a209bde90286 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:11.283174 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-468a54d4-2aea-42ae-955e-a209bde90286 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:21:11.284452 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-468a54d4-2aea-42ae-955e-a209bde90286 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:21:11.361600 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1215/2427] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:21:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:21:11.369935 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0576b45d-2037-4bf4-803f-ba1fc9a85611 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:11.370236 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0576b45d-2037-4bf4-803f-ba1fc9a85611 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:11.370522 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0576b45d-2037-4bf4-803f-ba1fc9a85611 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:11.391313 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-0576b45d-2037-4bf4-803f-ba1fc9a85611 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:21:11.392494 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-0576b45d-2037-4bf4-803f-ba1fc9a85611 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:21:11.400973 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1213/2428] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:21:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:21:11.405493 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468a54d4-2aea-42ae-955e-a209bde90286 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:21:11.415835 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468a54d4-2aea-42ae-955e-a209bde90286 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:21:11.420464 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af66ef82-d74f-41ec-bacd-40df9a7401b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:21:11.421115 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af66ef82-d74f-41ec-bacd-40df9a7401b3 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:11.421406 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af66ef82-d74f-41ec-bacd-40df9a7401b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:11.421691 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af66ef82-d74f-41ec-bacd-40df9a7401b3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:11.422426 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af66ef82-d74f-41ec-bacd-40df9a7401b3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:21:11.422739 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af66ef82-d74f-41ec-bacd-40df9a7401b3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:21:11.424371 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af66ef82-d74f-41ec-bacd-40df9a7401b3 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:21:11.436611 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1216/2429] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Jul 25 12:21:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 12:21:11.441497 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0576b45d-2037-4bf4-803f-ba1fc9a85611 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:21:11.451759 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0576b45d-2037-4bf4-803f-ba1fc9a85611 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:21:11.457250 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8566ea9-d326-4626-b7f4-ae1b099359ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:21:11.457931 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8566ea9-d326-4626-b7f4-ae1b099359ab admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:11.458231 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8566ea9-d326-4626-b7f4-ae1b099359ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:11.458498 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8566ea9-d326-4626-b7f4-ae1b099359ab admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:11.459207 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8566ea9-d326-4626-b7f4-ae1b099359ab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:21:11.461202 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8566ea9-d326-4626-b7f4-ae1b099359ab admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:21:11.493972 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1214/2430] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Jul 25 12:21:11 2022] POST /identity/v3/projects => generated 379 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:21:11.495092 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af66ef82-d74f-41ec-bacd-40df9a7401b3 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:21:11.506499 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af66ef82-d74f-41ec-bacd-40df9a7401b3 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:21:11.570766 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb112d3-c101-4c2e-b997-184cd97967ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:21:11.571443 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb112d3-c101-4c2e-b997-184cd97967ca admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:11.571767 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb112d3-c101-4c2e-b997-184cd97967ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:11.572051 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb112d3-c101-4c2e-b997-184cd97967ca admin admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:11.572761 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb112d3-c101-4c2e-b997-184cd97967ca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:21:11.574730 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb112d3-c101-4c2e-b997-184cd97967ca admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:21:11.595706 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1217/2431] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Jul 25 12:21:11 2022] POST /identity/v3/users => generated 290 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:21:11.600343 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8566ea9-d326-4626-b7f4-ae1b099359ab admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:21:11.611411 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8566ea9-d326-4626-b7f4-ae1b099359ab admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:21:11.616249 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3cf752-fbc8-46e2-af94-506d5a0b13c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:21:11.616980 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3cf752-fbc8-46e2-af94-506d5a0b13c8 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:11.617271 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3cf752-fbc8-46e2-af94-506d5a0b13c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:11.617581 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3cf752-fbc8-46e2-af94-506d5a0b13c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:11.618838 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3cf752-fbc8-46e2-af94-506d5a0b13c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:21:11.620475 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3cf752-fbc8-46e2-af94-506d5a0b13c8 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:21:11.631252 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1215/2432] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Jul 25 12:21:11 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:21:11.638323 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb112d3-c101-4c2e-b997-184cd97967ca admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:21:11.653459 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb112d3-c101-4c2e-b997-184cd97967ca admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:21:11.658527 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5708457-681d-45a9-a9ff-1e1b76619a12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:21:11.659193 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5708457-681d-45a9-a9ff-1e1b76619a12 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:11.659484 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5708457-681d-45a9-a9ff-1e1b76619a12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:11.659804 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5708457-681d-45a9-a9ff-1e1b76619a12 admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:11.660534 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5708457-681d-45a9-a9ff-1e1b76619a12 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:21:11.662139 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5708457-681d-45a9-a9ff-1e1b76619a12 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:21:11.676476 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1218/2433] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Jul 25 12:21:11 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:21:11.682287 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3cf752-fbc8-46e2-af94-506d5a0b13c8 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:21:11.692782 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3cf752-fbc8-46e2-af94-506d5a0b13c8 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:21:11.697570 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e9c994-ba63-442f-9bfe-0d160adddb5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:21:11.698598 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e9c994-ba63-442f-9bfe-0d160adddb5a admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:11.698895 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e9c994-ba63-442f-9bfe-0d160adddb5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:11.699182 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e9c994-ba63-442f-9bfe-0d160adddb5a admin admin] PATH_INFO: `/v3/projects/1819b878ca21468fbedf507f2b61942e/users/939fed61adfc4b349039e31f030c44d8/roles/472313ae69174578824df0bed6390744` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:11.713869 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e9c994-ba63-442f-9bfe-0d160adddb5a admin admin] RBAC: Authorizing `identity:create_grant(project_id=1819b878ca21468fbedf507f2b61942e, user_id=939fed61adfc4b349039e31f030c44d8, role_id=472313ae69174578824df0bed6390744)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:21:11.716573 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e9c994-ba63-442f-9bfe-0d160adddb5a admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:21:11.729736 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1216/2434] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Jul 25 12:21:11 2022] PUT /identity/v3/projects/1819b878ca21468fbedf507f2b61942e/users/939fed61adfc4b349039e31f030c44d8/roles/472313ae69174578824df0bed6390744 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:21:11.742128 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462e7121-45a1-4584-a5bc-0e576646686e None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:11.742424 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462e7121-45a1-4584-a5bc-0e576646686e None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:11.742764 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462e7121-45a1-4584-a5bc-0e576646686e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:11.763834 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:21:11.777423 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-462e7121-45a1-4584-a5bc-0e576646686e None None] MFA Rules not processed for user `939fed61adfc4b349039e31f030c44d8`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:21:11.778702 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-462e7121-45a1-4584-a5bc-0e576646686e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:21:11.845600 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1219/2435] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:21:11 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:21:11.850116 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e9c994-ba63-442f-9bfe-0d160adddb5a admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:21:11.871635 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:21:11.918776 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e9c994-ba63-442f-9bfe-0d160adddb5a admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:21:11.923894 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77398c12-0299-4c2c-85c3-62091489de13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:21:11.924570 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77398c12-0299-4c2c-85c3-62091489de13 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:11.924861 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77398c12-0299-4c2c-85c3-62091489de13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:11.925148 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77398c12-0299-4c2c-85c3-62091489de13 admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:11.925844 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77398c12-0299-4c2c-85c3-62091489de13 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:21:11.927801 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77398c12-0299-4c2c-85c3-62091489de13 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:21:11.958510 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1217/2436] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Jul 25 12:21:11 2022] POST /identity/v3/projects => generated 379 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:21:11.963449 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462e7121-45a1-4584-a5bc-0e576646686e None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:21:11.975353 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462e7121-45a1-4584-a5bc-0e576646686e None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:21:12.036854 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef7bac1-0465-4f2f-9cef-23c5f6d04d1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:21:12.037475 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef7bac1-0465-4f2f-9cef-23c5f6d04d1d admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:12.037813 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef7bac1-0465-4f2f-9cef-23c5f6d04d1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:12.038097 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef7bac1-0465-4f2f-9cef-23c5f6d04d1d admin admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:12.038755 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef7bac1-0465-4f2f-9cef-23c5f6d04d1d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:21:12.040737 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef7bac1-0465-4f2f-9cef-23c5f6d04d1d admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:21:12.062203 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1220/2437] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Jul 25 12:21:11 2022] POST /identity/v3/users => generated 290 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:21:12.066258 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77398c12-0299-4c2c-85c3-62091489de13 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:21:12.077134 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77398c12-0299-4c2c-85c3-62091489de13 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:21:12.082158 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12dacce-a92c-4ec2-9160-32f5c062c8d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:21:12.082899 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12dacce-a92c-4ec2-9160-32f5c062c8d7 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:12.083165 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12dacce-a92c-4ec2-9160-32f5c062c8d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:12.083444 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12dacce-a92c-4ec2-9160-32f5c062c8d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:12.084257 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12dacce-a92c-4ec2-9160-32f5c062c8d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:21:12.085874 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12dacce-a92c-4ec2-9160-32f5c062c8d7 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:21:12.099868 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1218/2438] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Jul 25 12:21:12 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:21:12.102301 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef7bac1-0465-4f2f-9cef-23c5f6d04d1d admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:21:12.112602 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef7bac1-0465-4f2f-9cef-23c5f6d04d1d admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:21:12.117185 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcae46f5-5946-4fd0-9113-eb22bb496a80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:21:12.117837 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcae46f5-5946-4fd0-9113-eb22bb496a80 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:12.118124 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcae46f5-5946-4fd0-9113-eb22bb496a80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:12.118403 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcae46f5-5946-4fd0-9113-eb22bb496a80 admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:12.119129 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcae46f5-5946-4fd0-9113-eb22bb496a80 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:21:12.120789 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcae46f5-5946-4fd0-9113-eb22bb496a80 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:21:12.130457 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1221/2439] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Jul 25 12:21:12 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:21:12.135445 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12dacce-a92c-4ec2-9160-32f5c062c8d7 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:21:12.145709 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12dacce-a92c-4ec2-9160-32f5c062c8d7 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:21:12.151334 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312b68bc-6965-4d66-b0e2-82f819f002a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:21:12.152324 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312b68bc-6965-4d66-b0e2-82f819f002a3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:12.152611 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312b68bc-6965-4d66-b0e2-82f819f002a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:12.152891 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312b68bc-6965-4d66-b0e2-82f819f002a3 admin admin] PATH_INFO: `/v3/projects/018f644136b84651a6f008d83c7c7a1b/users/b9fdd72fc39a4660aa764d942a471945/roles/472313ae69174578824df0bed6390744` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:12.168677 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312b68bc-6965-4d66-b0e2-82f819f002a3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=018f644136b84651a6f008d83c7c7a1b, user_id=b9fdd72fc39a4660aa764d942a471945, role_id=472313ae69174578824df0bed6390744)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:21:12.171397 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312b68bc-6965-4d66-b0e2-82f819f002a3 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:21:12.184337 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1219/2440] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Jul 25 12:21:12 2022] PUT /identity/v3/projects/018f644136b84651a6f008d83c7c7a1b/users/b9fdd72fc39a4660aa764d942a471945/roles/472313ae69174578824df0bed6390744 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 12:21:12.194595 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d3daed-37e1-4d4b-9821-3a7105e3815e None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:12.194890 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d3daed-37e1-4d4b-9821-3a7105e3815e None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:12.195172 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d3daed-37e1-4d4b-9821-3a7105e3815e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:12.223775 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-c6d3daed-37e1-4d4b-9821-3a7105e3815e None None] MFA Rules not processed for user `b9fdd72fc39a4660aa764d942a471945`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:21:12.224982 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-c6d3daed-37e1-4d4b-9821-3a7105e3815e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:21:12.292998 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1222/2441] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:21:12 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:21:12.298301 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312b68bc-6965-4d66-b0e2-82f819f002a3 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:21:12.366482 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312b68bc-6965-4d66-b0e2-82f819f002a3 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:21:12.372291 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e82e0bc-cf7d-4d46-bc46-962d6b6f6ecd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:21:12.373241 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e82e0bc-cf7d-4d46-bc46-962d6b6f6ecd admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:12.373531 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e82e0bc-cf7d-4d46-bc46-962d6b6f6ecd admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:12.373817 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e82e0bc-cf7d-4d46-bc46-962d6b6f6ecd admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:12.374784 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e82e0bc-cf7d-4d46-bc46-962d6b6f6ecd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:21:12.376780 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e82e0bc-cf7d-4d46-bc46-962d6b6f6ecd admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:21:12.410586 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1220/2442] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Jul 25 12:21:12 2022] POST /identity/v3/projects => generated 379 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:21:12.412665 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d3daed-37e1-4d4b-9821-3a7105e3815e None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:21:12.430746 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d3daed-37e1-4d4b-9821-3a7105e3815e None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:21:12.435581 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:21:12.438344 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e82e0bc-cf7d-4d46-bc46-962d6b6f6ecd admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:21:12.440297 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-0e82e0bc-cf7d-4d46-bc46-962d6b6f6ecd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:21:12.495629 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4b6b13-3557-42da-9dd9-356230d6f033 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:21:12.496426 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4b6b13-3557-42da-9dd9-356230d6f033 admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:12.496721 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4b6b13-3557-42da-9dd9-356230d6f033 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:12.497010 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4b6b13-3557-42da-9dd9-356230d6f033 admin admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:12.497716 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4b6b13-3557-42da-9dd9-356230d6f033 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:21:12.499676 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4b6b13-3557-42da-9dd9-356230d6f033 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:21:12.502158 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e82e0bc-cf7d-4d46-bc46-962d6b6f6ecd admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:21:12.505804 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4851d17-f2e4-42cd-89d1-fb015a0b5b85 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51733781201243e8abf9eca39b207750', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:21:12.506446 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4851d17-f2e4-42cd-89d1-fb015a0b5b85 service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:12.506736 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4851d17-f2e4-42cd-89d1-fb015a0b5b85 service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:12.507016 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4851d17-f2e4-42cd-89d1-fb015a0b5b85 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:12.509407 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-d4851d17-f2e4-42cd-89d1-fb015a0b5b85 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:21:12.522665 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4851d17-f2e4-42cd-89d1-fb015a0b5b85 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:21:12.524187 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4851d17-f2e4-42cd-89d1-fb015a0b5b85 service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:21:12.528506 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1223/2443] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Jul 25 12:21:12 2022] POST /identity/v3/users => generated 296 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 12:21:12.529012 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:21:12.533334 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4b6b13-3557-42da-9dd9-356230d6f033 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:21:12.536554 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1221/2444] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:21:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:21:12.544238 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4b6b13-3557-42da-9dd9-356230d6f033 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:21:12.552841 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ea6237-917e-4982-9dcd-31381b0951ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:21:12.553538 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ea6237-917e-4982-9dcd-31381b0951ed admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:12.553857 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ea6237-917e-4982-9dcd-31381b0951ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:12.554113 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ea6237-917e-4982-9dcd-31381b0951ed admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:12.555476 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ea6237-917e-4982-9dcd-31381b0951ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:21:12.557233 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ea6237-917e-4982-9dcd-31381b0951ed admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:21:12.567863 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1224/2445] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Jul 25 12:21:12 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:21:12.572831 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4851d17-f2e4-42cd-89d1-fb015a0b5b85 service neutron] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:21:12.583696 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4851d17-f2e4-42cd-89d1-fb015a0b5b85 service neutron] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:21:12.588904 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cabd214-1297-4e09-b534-30cd9215b9a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:21:12.589862 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cabd214-1297-4e09-b534-30cd9215b9a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:12.590147 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cabd214-1297-4e09-b534-30cd9215b9a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:12.590782 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cabd214-1297-4e09-b534-30cd9215b9a5 admin admin] PATH_INFO: `/v3/projects/081ac1f45bb240938c13a32c46d2368c/users/fe7205c6ce5349678ef8210df616d599/roles/a7d20885b6d94928a34b3cc5b3a00006` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:12.606520 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cabd214-1297-4e09-b534-30cd9215b9a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=081ac1f45bb240938c13a32c46d2368c, user_id=fe7205c6ce5349678ef8210df616d599, role_id=a7d20885b6d94928a34b3cc5b3a00006)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:21:12.609222 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cabd214-1297-4e09-b534-30cd9215b9a5 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:21:12.623917 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1222/2446] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Jul 25 12:21:12 2022] PUT /identity/v3/projects/081ac1f45bb240938c13a32c46d2368c/users/fe7205c6ce5349678ef8210df616d599/roles/a7d20885b6d94928a34b3cc5b3a00006 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:21:12.627978 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ea6237-917e-4982-9dcd-31381b0951ed admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:21:12.639657 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ea6237-917e-4982-9dcd-31381b0951ed admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:21:12.704570 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48cea8c-c900-4df6-8f8f-e19e3a4e700d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:21:12.705259 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48cea8c-c900-4df6-8f8f-e19e3a4e700d admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:12.705552 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48cea8c-c900-4df6-8f8f-e19e3a4e700d admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:12.705839 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48cea8c-c900-4df6-8f8f-e19e3a4e700d admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:12.706584 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48cea8c-c900-4df6-8f8f-e19e3a4e700d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:21:12.708282 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48cea8c-c900-4df6-8f8f-e19e3a4e700d admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:21:12.718912 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1225/2447] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Jul 25 12:21:12 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:21:12.723028 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cabd214-1297-4e09-b534-30cd9215b9a5 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:21:12.736250 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cabd214-1297-4e09-b534-30cd9215b9a5 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:21:12.741157 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0c64b2-cd0b-4c22-aed3-979d7abec6de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:21:12.742054 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0c64b2-cd0b-4c22-aed3-979d7abec6de admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:12.742378 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0c64b2-cd0b-4c22-aed3-979d7abec6de admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:12.743175 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0c64b2-cd0b-4c22-aed3-979d7abec6de admin admin] PATH_INFO: `/v3/projects/081ac1f45bb240938c13a32c46d2368c/users/fe7205c6ce5349678ef8210df616d599/roles/a7d20885b6d94928a34b3cc5b3a00006` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:12.745226 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0c64b2-cd0b-4c22-aed3-979d7abec6de admin admin] RBAC: Authorizing `identity:create_grant(project_id=081ac1f45bb240938c13a32c46d2368c, user_id=fe7205c6ce5349678ef8210df616d599, role_id=a7d20885b6d94928a34b3cc5b3a00006)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:21:12.747914 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0c64b2-cd0b-4c22-aed3-979d7abec6de admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:21:12.759652 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1223/2448] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Jul 25 12:21:12 2022] PUT /identity/v3/projects/081ac1f45bb240938c13a32c46d2368c/users/fe7205c6ce5349678ef8210df616d599/roles/a7d20885b6d94928a34b3cc5b3a00006 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:21:12.771691 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6f12cd-2294-4f16-9e74-3930473f3892 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:12.773441 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6f12cd-2294-4f16-9e74-3930473f3892 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:12.773786 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6f12cd-2294-4f16-9e74-3930473f3892 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:12.809221 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-fe6f12cd-2294-4f16-9e74-3930473f3892 None None] MFA Rules not processed for user `fe7205c6ce5349678ef8210df616d599`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:21:12.810472 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-fe6f12cd-2294-4f16-9e74-3930473f3892 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:21:12.891493 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1226/2449] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:21:12 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:21:12.903498 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0c64b2-cd0b-4c22-aed3-979d7abec6de admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:21:12.905435 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-bb0c64b2-cd0b-4c22-aed3-979d7abec6de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:21:12.969432 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0c64b2-cd0b-4c22-aed3-979d7abec6de admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:21:12.973111 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ea550a-eee8-444e-86bd-35f732c3fa81 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f89ce6f0cf04373a414d927fa0f8a7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:21:12.973767 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ea550a-eee8-444e-86bd-35f732c3fa81 service cinder] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:12.974066 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ea550a-eee8-444e-86bd-35f732c3fa81 service cinder] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:12.974351 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ea550a-eee8-444e-86bd-35f732c3fa81 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:12.985017 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ea550a-eee8-444e-86bd-35f732c3fa81 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:21:12.986467 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ea550a-eee8-444e-86bd-35f732c3fa81 service cinder] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:21:12.998438 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1224/2450] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:21:12 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:21:13.043294 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6f12cd-2294-4f16-9e74-3930473f3892 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:21:13.056640 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6f12cd-2294-4f16-9e74-3930473f3892 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:21:13.067758 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994806a9-c839-4d6e-a1f4-a584bbff31dd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f89ce6f0cf04373a414d927fa0f8a7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:21:13.068609 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994806a9-c839-4d6e-a1f4-a584bbff31dd service cinder] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:13.069474 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994806a9-c839-4d6e-a1f4-a584bbff31dd service cinder] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:13.069815 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994806a9-c839-4d6e-a1f4-a584bbff31dd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:13.076195 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:21:13.136945 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994806a9-c839-4d6e-a1f4-a584bbff31dd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:21:13.138872 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994806a9-c839-4d6e-a1f4-a584bbff31dd service cinder] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:21:13.149609 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1227/2451] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:21:13 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:21:15.886239 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ea550a-eee8-444e-86bd-35f732c3fa81 service cinder] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:21:15.887782 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-93ea550a-eee8-444e-86bd-35f732c3fa81 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:21:15.956381 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ea550a-eee8-444e-86bd-35f732c3fa81 service cinder] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:21:15.960975 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1be4bc-967e-413a-ac29-c3819ee2bc67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:21:15.961843 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1be4bc-967e-413a-ac29-c3819ee2bc67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:15.962131 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1be4bc-967e-413a-ac29-c3819ee2bc67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:15.962412 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1be4bc-967e-413a-ac29-c3819ee2bc67 admin admin] PATH_INFO: `/v3/users/3848eae5b8df48eb9816ba0d7485f7c0` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:15.963635 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1be4bc-967e-413a-ac29-c3819ee2bc67 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3848eae5b8df48eb9816ba0d7485f7c0)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:21:15.965561 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1be4bc-967e-413a-ac29-c3819ee2bc67 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:21:16.010957 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-7c1be4bc-967e-413a-ac29-c3819ee2bc67 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3848eae5b8df48eb9816ba0d7485f7c0'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:16.018225 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-7c1be4bc-967e-413a-ac29-c3819ee2bc67 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3848eae5b8df48eb9816ba0d7485f7c0'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:16.027837 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-7c1be4bc-967e-413a-ac29-c3819ee2bc67 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3848eae5b8df48eb9816ba0d7485f7c0'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:16.030250 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=89,keystone:INSERT=3,keystone:DELETE=8 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:21:16.043224 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-7c1be4bc-967e-413a-ac29-c3819ee2bc67 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3848eae5b8df48eb9816ba0d7485f7c0'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:16.043885 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-7c1be4bc-967e-413a-ac29-c3819ee2bc67 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3848eae5b8df48eb9816ba0d7485f7c0'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:16.051885 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1225/2452] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Jul 25 12:21:15 2022] DELETE /identity/v3/users/3848eae5b8df48eb9816ba0d7485f7c0 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:21:16.137652 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994806a9-c839-4d6e-a1f4-a584bbff31dd service cinder] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:21:16.148396 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994806a9-c839-4d6e-a1f4-a584bbff31dd service cinder] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:21:16.213349 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51431470-757e-4851-bbb2-c1ad80894cc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:21:16.214249 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51431470-757e-4851-bbb2-c1ad80894cc3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:16.214536 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51431470-757e-4851-bbb2-c1ad80894cc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:16.214819 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51431470-757e-4851-bbb2-c1ad80894cc3 admin admin] PATH_INFO: `/v3/users/939fed61adfc4b349039e31f030c44d8` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:16.216657 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51431470-757e-4851-bbb2-c1ad80894cc3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=939fed61adfc4b349039e31f030c44d8)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:21:16.218149 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51431470-757e-4851-bbb2-c1ad80894cc3 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:21:16.260772 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1be4bc-967e-413a-ac29-c3819ee2bc67 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:21:16.264431 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-7c1be4bc-967e-413a-ac29-c3819ee2bc67 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:21:16.284129 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1be4bc-967e-413a-ac29-c3819ee2bc67 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:21:16.289012 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25827953-16a7-449c-9d88-896f9a0d8c88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:21:16.290453 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25827953-16a7-449c-9d88-896f9a0d8c88 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:16.290775 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25827953-16a7-449c-9d88-896f9a0d8c88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:16.291003 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25827953-16a7-449c-9d88-896f9a0d8c88 admin admin] PATH_INFO: `/v3/projects/11ff6cbb1ad04f0faa8b2ed2a66cdca4` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:16.292234 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25827953-16a7-449c-9d88-896f9a0d8c88 admin admin] RBAC: Authorizing `identity:delete_project(project_id=11ff6cbb1ad04f0faa8b2ed2a66cdca4)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:21:16.294271 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25827953-16a7-449c-9d88-896f9a0d8c88 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:21:16.297946 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-51431470-757e-4851-bbb2-c1ad80894cc3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '939fed61adfc4b349039e31f030c44d8'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:16.298723 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-51431470-757e-4851-bbb2-c1ad80894cc3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '939fed61adfc4b349039e31f030c44d8'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:16.307275 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-51431470-757e-4851-bbb2-c1ad80894cc3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '939fed61adfc4b349039e31f030c44d8'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:16.323592 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-51431470-757e-4851-bbb2-c1ad80894cc3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '939fed61adfc4b349039e31f030c44d8'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:16.328504 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-51431470-757e-4851-bbb2-c1ad80894cc3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '939fed61adfc4b349039e31f030c44d8'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:16.337327 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1228/2453] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Jul 25 12:21:16 2022] DELETE /identity/v3/users/939fed61adfc4b349039e31f030c44d8 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:21:16.341562 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d31b0a-7888-49c7-847c-400592906eb5 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:16.341863 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d31b0a-7888-49c7-847c-400592906eb5 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:16.342198 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d31b0a-7888-49c7-847c-400592906eb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:16.353221 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-25827953-16a7-449c-9d88-896f9a0d8c88 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '11ff6cbb1ad04f0faa8b2ed2a66cdca4', 'request_id': 'req-25827953-16a7-449c-9d88-896f9a0d8c88'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:16.360734 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-25827953-16a7-449c-9d88-896f9a0d8c88 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '11ff6cbb1ad04f0faa8b2ed2a66cdca4', 'request_id': 'req-25827953-16a7-449c-9d88-896f9a0d8c88'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:16.362257 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-97d31b0a-7888-49c7-847c-400592906eb5 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:21:16.363475 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-97d31b0a-7888-49c7-847c-400592906eb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:21:16.366854 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-25827953-16a7-449c-9d88-896f9a0d8c88 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '11ff6cbb1ad04f0faa8b2ed2a66cdca4', 'request_id': 'req-25827953-16a7-449c-9d88-896f9a0d8c88'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:16.380607 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-25827953-16a7-449c-9d88-896f9a0d8c88 admin admin] The token cache is being invalidate because project 11ff6cbb1ad04f0faa8b2ed2a66cdca4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 12:21:16.380908 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-25827953-16a7-449c-9d88-896f9a0d8c88 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:16.381578 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-25827953-16a7-449c-9d88-896f9a0d8c88 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:16.383176 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1226/2454] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Jul 25 12:21:16 2022] DELETE /identity/v3/projects/11ff6cbb1ad04f0faa8b2ed2a66cdca4 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:21:16.388303 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5066eeba-639d-4595-b0f0-9e0bb3d0bc21 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:16.388662 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5066eeba-639d-4595-b0f0-9e0bb3d0bc21 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:16.389007 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5066eeba-639d-4595-b0f0-9e0bb3d0bc21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:16.391347 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:21:16.411606 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-5066eeba-639d-4595-b0f0-9e0bb3d0bc21 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:21:16.412821 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-5066eeba-639d-4595-b0f0-9e0bb3d0bc21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:21:16.435564 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1229/2455] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:21:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 12:21:16.446442 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d31b0a-7888-49c7-847c-400592906eb5 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:21:16.447821 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-97d31b0a-7888-49c7-847c-400592906eb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:21:16.475364 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1227/2456] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:21:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 12:21:16.485939 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01348bac-932b-4110-bfb2-1f1f340c9960 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:16.486302 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01348bac-932b-4110-bfb2-1f1f340c9960 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:16.486556 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01348bac-932b-4110-bfb2-1f1f340c9960 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:16.506043 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-01348bac-932b-4110-bfb2-1f1f340c9960 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:21:16.507242 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-01348bac-932b-4110-bfb2-1f1f340c9960 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:21:16.509282 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d31b0a-7888-49c7-847c-400592906eb5 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:21:16.513123 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9890bff-a0b2-4a86-b035-251246b8865a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51733781201243e8abf9eca39b207750', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:21:16.513775 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9890bff-a0b2-4a86-b035-251246b8865a service neutron] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:16.514065 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9890bff-a0b2-4a86-b035-251246b8865a service neutron] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:16.514370 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9890bff-a0b2-4a86-b035-251246b8865a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:16.515714 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1228/2457] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:21:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 12:21:16.519674 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01348bac-932b-4110-bfb2-1f1f340c9960 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:21:16.524439 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9890bff-a0b2-4a86-b035-251246b8865a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:21:16.526367 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9890bff-a0b2-4a86-b035-251246b8865a service neutron] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:21:16.528928 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01348bac-932b-4110-bfb2-1f1f340c9960 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:21:16.536975 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1230/2458] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:21:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:21:16.537674 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe58fcc-188e-4227-8d73-40f6a0d18862 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:21:16.538535 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe58fcc-188e-4227-8d73-40f6a0d18862 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:16.538895 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe58fcc-188e-4227-8d73-40f6a0d18862 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:16.540356 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe58fcc-188e-4227-8d73-40f6a0d18862 admin admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:16.541178 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe58fcc-188e-4227-8d73-40f6a0d18862 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:21:16.541492 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe58fcc-188e-4227-8d73-40f6a0d18862 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:21:16.543111 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe58fcc-188e-4227-8d73-40f6a0d18862 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:21:16.556598 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1229/2459] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Jul 25 12:21:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 12:21:16.561105 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9890bff-a0b2-4a86-b035-251246b8865a service neutron] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:21:16.571008 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9890bff-a0b2-4a86-b035-251246b8865a service neutron] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:21:16.575786 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becb2ed9-bc4a-44b2-bc11-3fffd6ffd21b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:21:16.576459 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becb2ed9-bc4a-44b2-bc11-3fffd6ffd21b admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:16.576747 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becb2ed9-bc4a-44b2-bc11-3fffd6ffd21b admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:16.577031 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becb2ed9-bc4a-44b2-bc11-3fffd6ffd21b admin admin] PATH_INFO: `/v3/projects` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:16.577731 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-becb2ed9-bc4a-44b2-bc11-3fffd6ffd21b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:21:16.579678 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-becb2ed9-bc4a-44b2-bc11-3fffd6ffd21b admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:21:16.610370 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1231/2460] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Jul 25 12:21:16 2022] POST /identity/v3/projects => generated 375 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:21:16.614600 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe58fcc-188e-4227-8d73-40f6a0d18862 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:21:16.625916 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe58fcc-188e-4227-8d73-40f6a0d18862 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:21:16.682616 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:21:16.701637 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93eec919-a639-4012-b7ef-ec124d8aa0d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:21:16.702313 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93eec919-a639-4012-b7ef-ec124d8aa0d9 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:16.702734 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93eec919-a639-4012-b7ef-ec124d8aa0d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:16.703019 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93eec919-a639-4012-b7ef-ec124d8aa0d9 admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:16.703758 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93eec919-a639-4012-b7ef-ec124d8aa0d9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:21:16.705861 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93eec919-a639-4012-b7ef-ec124d8aa0d9 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:21:16.735592 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1230/2461] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Jul 25 12:21:16 2022] POST /identity/v3/users => generated 288 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 12:21:16.738850 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becb2ed9-bc4a-44b2-bc11-3fffd6ffd21b admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:21:16.748948 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becb2ed9-bc4a-44b2-bc11-3fffd6ffd21b admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:21:16.754023 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1eb3d5f-609b-4659-8bea-e28cc54c939e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:21:16.754699 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1eb3d5f-609b-4659-8bea-e28cc54c939e admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:16.754986 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1eb3d5f-609b-4659-8bea-e28cc54c939e admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:16.755266 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1eb3d5f-609b-4659-8bea-e28cc54c939e admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:16.756134 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1eb3d5f-609b-4659-8bea-e28cc54c939e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:21:16.757626 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1eb3d5f-609b-4659-8bea-e28cc54c939e admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:21:16.787584 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1232/2462] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Jul 25 12:21:16 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:21:16.792425 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93eec919-a639-4012-b7ef-ec124d8aa0d9 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:21:16.808841 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93eec919-a639-4012-b7ef-ec124d8aa0d9 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:21:16.819713 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e876d39b-d04a-4fa0-9c89-a94cac0502dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:21:16.826522 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e876d39b-d04a-4fa0-9c89-a94cac0502dd admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:16.827877 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e876d39b-d04a-4fa0-9c89-a94cac0502dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:16.828225 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e876d39b-d04a-4fa0-9c89-a94cac0502dd admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:16.829046 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e876d39b-d04a-4fa0-9c89-a94cac0502dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:21:16.831647 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e876d39b-d04a-4fa0-9c89-a94cac0502dd admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:21:16.848857 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1231/2463] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Jul 25 12:21:16 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:21:16.849305 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1eb3d5f-609b-4659-8bea-e28cc54c939e admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:21:16.850468 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-f1eb3d5f-609b-4659-8bea-e28cc54c939e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:21:16.852742 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e876d39b-d04a-4fa0-9c89-a94cac0502dd admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:21:16.864934 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1eb3d5f-609b-4659-8bea-e28cc54c939e admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:21:16.865085 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e876d39b-d04a-4fa0-9c89-a94cac0502dd admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:21:16.871058 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee44b042-7863-41a1-8076-a7cf03582607 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:21:16.872081 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee44b042-7863-41a1-8076-a7cf03582607 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:16.872887 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee44b042-7863-41a1-8076-a7cf03582607 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:16.873147 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ca8312-fa41-4334-8610-e4c12cbb6eb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:21:16.873277 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee44b042-7863-41a1-8076-a7cf03582607 admin admin] PATH_INFO: `/v3/projects/1819b878ca21468fbedf507f2b61942e` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:16.874198 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ca8312-fa41-4334-8610-e4c12cbb6eb0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:16.874555 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ca8312-fa41-4334-8610-e4c12cbb6eb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:16.874719 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee44b042-7863-41a1-8076-a7cf03582607 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1819b878ca21468fbedf507f2b61942e)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:21:16.875025 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ca8312-fa41-4334-8610-e4c12cbb6eb0 admin admin] PATH_INFO: `/v3/projects/7deb5e10b1a84db5bb34b6059fff861c/users/7c6602c69fad4e86a96e4c2843c4f5f7/roles/472313ae69174578824df0bed6390744` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:16.881613 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee44b042-7863-41a1-8076-a7cf03582607 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:21:16.890648 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ca8312-fa41-4334-8610-e4c12cbb6eb0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7deb5e10b1a84db5bb34b6059fff861c, user_id=7c6602c69fad4e86a96e4c2843c4f5f7, role_id=472313ae69174578824df0bed6390744)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:21:16.893488 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ca8312-fa41-4334-8610-e4c12cbb6eb0 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:21:16.907727 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1232/2464] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Jul 25 12:21:16 2022] PUT /identity/v3/projects/7deb5e10b1a84db5bb34b6059fff861c/users/7c6602c69fad4e86a96e4c2843c4f5f7/roles/472313ae69174578824df0bed6390744 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:21:16.919770 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1aed7f9-3364-422b-81bd-f60a3c3ebf48 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:16.920053 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1aed7f9-3364-422b-81bd-f60a3c3ebf48 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:16.920341 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1aed7f9-3364-422b-81bd-f60a3c3ebf48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:16.934213 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-ee44b042-7863-41a1-8076-a7cf03582607 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1819b878ca21468fbedf507f2b61942e', 'request_id': 'req-ee44b042-7863-41a1-8076-a7cf03582607'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:16.943199 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-ee44b042-7863-41a1-8076-a7cf03582607 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1819b878ca21468fbedf507f2b61942e', 'request_id': 'req-ee44b042-7863-41a1-8076-a7cf03582607'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:16.948948 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-ee44b042-7863-41a1-8076-a7cf03582607 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1819b878ca21468fbedf507f2b61942e', 'request_id': 'req-ee44b042-7863-41a1-8076-a7cf03582607'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:16.949098 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-e1aed7f9-3364-422b-81bd-f60a3c3ebf48 None None] MFA Rules not processed for user `7c6602c69fad4e86a96e4c2843c4f5f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:21:16.950223 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-e1aed7f9-3364-422b-81bd-f60a3c3ebf48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:21:16.959820 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-ee44b042-7863-41a1-8076-a7cf03582607 admin admin] The token cache is being invalidate because project 1819b878ca21468fbedf507f2b61942e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80298) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 12:21:16.960104 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-ee44b042-7863-41a1-8076-a7cf03582607 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:16.960789 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-ee44b042-7863-41a1-8076-a7cf03582607 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:16.962863 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1233/2465] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Jul 25 12:21:16 2022] DELETE /identity/v3/projects/1819b878ca21468fbedf507f2b61942e => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 12:21:16.967037 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee44b042-7863-41a1-8076-a7cf03582607 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:21:16.968467 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-ee44b042-7863-41a1-8076-a7cf03582607 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:21:16.990893 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:21:17.021106 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1233/2466] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:21:16 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 12:21:17.033315 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1aed7f9-3364-422b-81bd-f60a3c3ebf48 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:21:17.034846 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-e1aed7f9-3364-422b-81bd-f60a3c3ebf48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:21:17.041966 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee44b042-7863-41a1-8076-a7cf03582607 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:21:17.046532 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b02f5ae-fc4f-4616-a285-51a075cb5fd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:21:17.047520 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b02f5ae-fc4f-4616-a285-51a075cb5fd9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:17.047846 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b02f5ae-fc4f-4616-a285-51a075cb5fd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:17.048393 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b02f5ae-fc4f-4616-a285-51a075cb5fd9 admin admin] PATH_INFO: `/v3/users/b9fdd72fc39a4660aa764d942a471945` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:17.050096 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b02f5ae-fc4f-4616-a285-51a075cb5fd9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b9fdd72fc39a4660aa764d942a471945)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:21:17.052409 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b02f5ae-fc4f-4616-a285-51a075cb5fd9 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:21:17.101188 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-2b02f5ae-fc4f-4616-a285-51a075cb5fd9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b9fdd72fc39a4660aa764d942a471945'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:17.102358 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-2b02f5ae-fc4f-4616-a285-51a075cb5fd9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b9fdd72fc39a4660aa764d942a471945'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:17.109555 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-2b02f5ae-fc4f-4616-a285-51a075cb5fd9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b9fdd72fc39a4660aa764d942a471945'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:17.116325 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1aed7f9-3364-422b-81bd-f60a3c3ebf48 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:21:17.121743 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-2b02f5ae-fc4f-4616-a285-51a075cb5fd9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b9fdd72fc39a4660aa764d942a471945'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:17.122581 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ac6831-f94a-40e1-9db9-f0c97d8383c5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f67142a1193d47c7bd33d7040268233c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:21:17.123238 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ac6831-f94a-40e1-9db9-f0c97d8383c5 service nova] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:17.123529 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ac6831-f94a-40e1-9db9-f0c97d8383c5 service nova] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:17.123846 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ac6831-f94a-40e1-9db9-f0c97d8383c5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:17.128728 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-2b02f5ae-fc4f-4616-a285-51a075cb5fd9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b9fdd72fc39a4660aa764d942a471945'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:17.131878 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ac6831-f94a-40e1-9db9-f0c97d8383c5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:21:17.133576 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ac6831-f94a-40e1-9db9-f0c97d8383c5 service nova] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:21:17.134750 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1234/2467] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Jul 25 12:21:16 2022] DELETE /identity/v3/users/b9fdd72fc39a4660aa764d942a471945 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 12:21:17.142934 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1234/2468] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:21:17 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 12:21:17.394956 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b02f5ae-fc4f-4616-a285-51a075cb5fd9 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:21:17.396502 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-2b02f5ae-fc4f-4616-a285-51a075cb5fd9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:21:17.456371 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ac6831-f94a-40e1-9db9-f0c97d8383c5 service nova] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:21:17.470856 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ac6831-f94a-40e1-9db9-f0c97d8383c5 service nova] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:21:17.472010 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b02f5ae-fc4f-4616-a285-51a075cb5fd9 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:21:17.477275 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6910879-62fe-44fd-ab9e-dc436b2f0618 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:21:17.478407 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6910879-62fe-44fd-ab9e-dc436b2f0618 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:17.478538 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32f8189-3174-4e5b-ad62-05e61bd208d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:21:17.478783 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6910879-62fe-44fd-ab9e-dc436b2f0618 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:17.478972 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6910879-62fe-44fd-ab9e-dc436b2f0618 admin admin] PATH_INFO: `/v3/projects/018f644136b84651a6f008d83c7c7a1b` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:17.479359 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32f8189-3174-4e5b-ad62-05e61bd208d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:17.479693 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32f8189-3174-4e5b-ad62-05e61bd208d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:17.480183 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6910879-62fe-44fd-ab9e-dc436b2f0618 admin admin] RBAC: Authorizing `identity:delete_project(project_id=018f644136b84651a6f008d83c7c7a1b)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:21:17.480532 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32f8189-3174-4e5b-ad62-05e61bd208d5 admin admin] PATH_INFO: `/v3/users/7c6602c69fad4e86a96e4c2843c4f5f7` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:17.481837 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32f8189-3174-4e5b-ad62-05e61bd208d5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7c6602c69fad4e86a96e4c2843c4f5f7)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:21:17.482134 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6910879-62fe-44fd-ab9e-dc436b2f0618 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:21:17.484954 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32f8189-3174-4e5b-ad62-05e61bd208d5 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:21:17.494394 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:21:17.530975 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:21:17.531586 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-a6910879-62fe-44fd-ab9e-dc436b2f0618 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '018f644136b84651a6f008d83c7c7a1b', 'request_id': 'req-a6910879-62fe-44fd-ab9e-dc436b2f0618'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:17.537286 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-e32f8189-3174-4e5b-ad62-05e61bd208d5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7c6602c69fad4e86a96e4c2843c4f5f7'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:17.538028 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-e32f8189-3174-4e5b-ad62-05e61bd208d5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7c6602c69fad4e86a96e4c2843c4f5f7'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:17.542445 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-a6910879-62fe-44fd-ab9e-dc436b2f0618 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '018f644136b84651a6f008d83c7c7a1b', 'request_id': 'req-a6910879-62fe-44fd-ab9e-dc436b2f0618'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:17.547035 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-e32f8189-3174-4e5b-ad62-05e61bd208d5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7c6602c69fad4e86a96e4c2843c4f5f7'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:17.548799 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-a6910879-62fe-44fd-ab9e-dc436b2f0618 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '018f644136b84651a6f008d83c7c7a1b', 'request_id': 'req-a6910879-62fe-44fd-ab9e-dc436b2f0618'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:17.560310 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-e32f8189-3174-4e5b-ad62-05e61bd208d5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7c6602c69fad4e86a96e4c2843c4f5f7'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:17.561075 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-a6910879-62fe-44fd-ab9e-dc436b2f0618 admin admin] The token cache is being invalidate because project 018f644136b84651a6f008d83c7c7a1b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 12:21:17.561755 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-a6910879-62fe-44fd-ab9e-dc436b2f0618 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:17.562458 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-a6910879-62fe-44fd-ab9e-dc436b2f0618 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:17.564231 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1235/2469] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Jul 25 12:21:17 2022] DELETE /identity/v3/projects/018f644136b84651a6f008d83c7c7a1b => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:21:17.568017 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6910879-62fe-44fd-ab9e-dc436b2f0618 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:21:17.568167 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-e32f8189-3174-4e5b-ad62-05e61bd208d5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7c6602c69fad4e86a96e4c2843c4f5f7'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:17.569410 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-a6910879-62fe-44fd-ab9e-dc436b2f0618 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:21:17.574479 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1235/2470] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Jul 25 12:21:17 2022] DELETE /identity/v3/users/7c6602c69fad4e86a96e4c2843c4f5f7 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:21:17.578778 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e3a8ed-9acc-4a11-aae6-2ec0f145b4ed None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:17.579087 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e3a8ed-9acc-4a11-aae6-2ec0f145b4ed None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:17.579378 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e3a8ed-9acc-4a11-aae6-2ec0f145b4ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:17.597654 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-98e3a8ed-9acc-4a11-aae6-2ec0f145b4ed None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:21:17.598845 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-98e3a8ed-9acc-4a11-aae6-2ec0f145b4ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:21:17.634954 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6910879-62fe-44fd-ab9e-dc436b2f0618 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:21:17.659629 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1236/2471] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:21:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 12:21:17.669766 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e3a8ed-9acc-4a11-aae6-2ec0f145b4ed None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:21:17.671114 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-98e3a8ed-9acc-4a11-aae6-2ec0f145b4ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:21:17.693929 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db82b95c-a349-4873-b1ba-5d1464dd2e70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:21:17.694804 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db82b95c-a349-4873-b1ba-5d1464dd2e70 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:17.695088 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db82b95c-a349-4873-b1ba-5d1464dd2e70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:17.695369 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db82b95c-a349-4873-b1ba-5d1464dd2e70 admin admin] PATH_INFO: `/v3/users/fe7205c6ce5349678ef8210df616d599` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:17.696621 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db82b95c-a349-4873-b1ba-5d1464dd2e70 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fe7205c6ce5349678ef8210df616d599)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:21:17.698540 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db82b95c-a349-4873-b1ba-5d1464dd2e70 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:21:17.725982 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e3a8ed-9acc-4a11-aae6-2ec0f145b4ed None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:21:17.730758 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2132ad-9b9b-46df-913a-ff85515cdbc6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51733781201243e8abf9eca39b207750', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:21:17.731396 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2132ad-9b9b-46df-913a-ff85515cdbc6 service neutron] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:17.731723 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2132ad-9b9b-46df-913a-ff85515cdbc6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:17.733226 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2132ad-9b9b-46df-913a-ff85515cdbc6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:17.741441 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2132ad-9b9b-46df-913a-ff85515cdbc6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:21:17.742896 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2132ad-9b9b-46df-913a-ff85515cdbc6 service neutron] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:21:17.754126 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-db82b95c-a349-4873-b1ba-5d1464dd2e70 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fe7205c6ce5349678ef8210df616d599'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:17.755364 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:21:17.758439 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1237/2472] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:21:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 12:21:17.763689 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-db82b95c-a349-4873-b1ba-5d1464dd2e70 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fe7205c6ce5349678ef8210df616d599'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:17.771660 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-db82b95c-a349-4873-b1ba-5d1464dd2e70 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fe7205c6ce5349678ef8210df616d599'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:17.785424 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-db82b95c-a349-4873-b1ba-5d1464dd2e70 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fe7205c6ce5349678ef8210df616d599'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:17.786164 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-db82b95c-a349-4873-b1ba-5d1464dd2e70 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fe7205c6ce5349678ef8210df616d599'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:17.792854 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1236/2473] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Jul 25 12:21:17 2022] DELETE /identity/v3/users/fe7205c6ce5349678ef8210df616d599 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 12:21:18.124144 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2132ad-9b9b-46df-913a-ff85515cdbc6 service neutron] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:21:18.125648 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-2e2132ad-9b9b-46df-913a-ff85515cdbc6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:21:18.140819 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2132ad-9b9b-46df-913a-ff85515cdbc6 service neutron] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:21:18.180724 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db82b95c-a349-4873-b1ba-5d1464dd2e70 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:21:18.183739 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-db82b95c-a349-4873-b1ba-5d1464dd2e70 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:21:18.216382 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909a8662-419f-4e1d-81a1-ec2f50e76b29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:21:18.217265 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909a8662-419f-4e1d-81a1-ec2f50e76b29 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:18.217558 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909a8662-419f-4e1d-81a1-ec2f50e76b29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:18.217842 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909a8662-419f-4e1d-81a1-ec2f50e76b29 admin admin] PATH_INFO: `/v3/projects/7deb5e10b1a84db5bb34b6059fff861c` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:18.219110 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909a8662-419f-4e1d-81a1-ec2f50e76b29 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7deb5e10b1a84db5bb34b6059fff861c)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:21:18.221059 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909a8662-419f-4e1d-81a1-ec2f50e76b29 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:21:18.223507 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:21:18.255644 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db82b95c-a349-4873-b1ba-5d1464dd2e70 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:21:18.260924 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22f325b-3f16-4033-b5d1-f3988a8b5be6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:21:18.261783 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22f325b-3f16-4033-b5d1-f3988a8b5be6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:18.262234 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22f325b-3f16-4033-b5d1-f3988a8b5be6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:18.262389 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22f325b-3f16-4033-b5d1-f3988a8b5be6 admin admin] PATH_INFO: `/v3/projects/081ac1f45bb240938c13a32c46d2368c` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:18.263880 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22f325b-3f16-4033-b5d1-f3988a8b5be6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=081ac1f45bb240938c13a32c46d2368c)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:21:18.266134 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22f325b-3f16-4033-b5d1-f3988a8b5be6 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:21:18.273015 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-909a8662-419f-4e1d-81a1-ec2f50e76b29 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7deb5e10b1a84db5bb34b6059fff861c', 'request_id': 'req-909a8662-419f-4e1d-81a1-ec2f50e76b29'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:18.281433 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-909a8662-419f-4e1d-81a1-ec2f50e76b29 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7deb5e10b1a84db5bb34b6059fff861c', 'request_id': 'req-909a8662-419f-4e1d-81a1-ec2f50e76b29'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:18.287312 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-909a8662-419f-4e1d-81a1-ec2f50e76b29 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7deb5e10b1a84db5bb34b6059fff861c', 'request_id': 'req-909a8662-419f-4e1d-81a1-ec2f50e76b29'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:18.299334 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-909a8662-419f-4e1d-81a1-ec2f50e76b29 admin admin] The token cache is being invalidate because project 7deb5e10b1a84db5bb34b6059fff861c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80298) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 12:21:18.299655 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-909a8662-419f-4e1d-81a1-ec2f50e76b29 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:18.300752 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-909a8662-419f-4e1d-81a1-ec2f50e76b29 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:18.302708 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1238/2474] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Jul 25 12:21:18 2022] DELETE /identity/v3/projects/7deb5e10b1a84db5bb34b6059fff861c => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:21:18.307925 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6d4cd3-f7ba-4c26-9ae3-b5164c5130df None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:18.308220 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6d4cd3-f7ba-4c26-9ae3-b5164c5130df None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:18.308494 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6d4cd3-f7ba-4c26-9ae3-b5164c5130df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:18.315969 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-c22f325b-3f16-4033-b5d1-f3988a8b5be6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '081ac1f45bb240938c13a32c46d2368c', 'request_id': 'req-c22f325b-3f16-4033-b5d1-f3988a8b5be6'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:18.324152 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-c22f325b-3f16-4033-b5d1-f3988a8b5be6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '081ac1f45bb240938c13a32c46d2368c', 'request_id': 'req-c22f325b-3f16-4033-b5d1-f3988a8b5be6'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:18.329280 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-5a6d4cd3-f7ba-4c26-9ae3-b5164c5130df None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:21:18.330260 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-c22f325b-3f16-4033-b5d1-f3988a8b5be6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '081ac1f45bb240938c13a32c46d2368c', 'request_id': 'req-c22f325b-3f16-4033-b5d1-f3988a8b5be6'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:18.330502 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-5a6d4cd3-f7ba-4c26-9ae3-b5164c5130df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:21:18.341713 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-c22f325b-3f16-4033-b5d1-f3988a8b5be6 admin admin] The token cache is being invalidate because project 081ac1f45bb240938c13a32c46d2368c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 12:21:18.342009 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-c22f325b-3f16-4033-b5d1-f3988a8b5be6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:18.342664 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-c22f325b-3f16-4033-b5d1-f3988a8b5be6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:18.345507 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1237/2475] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Jul 25 12:21:18 2022] DELETE /identity/v3/projects/081ac1f45bb240938c13a32c46d2368c => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:21:18.350200 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f9644b-0a67-405b-8cb4-9e4fed142b3a None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:18.350656 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f9644b-0a67-405b-8cb4-9e4fed142b3a None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:18.350941 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f9644b-0a67-405b-8cb4-9e4fed142b3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:18.373615 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-a5f9644b-0a67-405b-8cb4-9e4fed142b3a None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:21:18.374868 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-a5f9644b-0a67-405b-8cb4-9e4fed142b3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:21:18.399023 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1239/2476] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:21:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 12:21:18.407091 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d755e0-43c9-4034-b969-86c90646caee None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:18.407571 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d755e0-43c9-4034-b969-86c90646caee None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:18.407876 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d755e0-43c9-4034-b969-86c90646caee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:18.426462 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:21:18.428018 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-54d755e0-43c9-4034-b969-86c90646caee None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:21:18.429457 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-54d755e0-43c9-4034-b969-86c90646caee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:21:18.438130 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1240/2477] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:21:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 12:21:18.439637 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1238/2478] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:21:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 12:21:18.442224 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d755e0-43c9-4034-b969-86c90646caee None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:21:18.447913 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dab280b-db18-4bb5-8925-df616bc01597 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:18.448270 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dab280b-db18-4bb5-8925-df616bc01597 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:18.448558 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dab280b-db18-4bb5-8925-df616bc01597 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:18.451523 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d755e0-43c9-4034-b969-86c90646caee None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:21:18.456183 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6266951-47ae-44a9-b74c-c8ec93037086 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:21:18.456832 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6266951-47ae-44a9-b74c-c8ec93037086 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:18.457123 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6266951-47ae-44a9-b74c-c8ec93037086 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:18.457411 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6266951-47ae-44a9-b74c-c8ec93037086 admin admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:18.458140 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6266951-47ae-44a9-b74c-c8ec93037086 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:21:18.458456 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6266951-47ae-44a9-b74c-c8ec93037086 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:21:18.460693 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6266951-47ae-44a9-b74c-c8ec93037086 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:21:18.468369 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-1dab280b-db18-4bb5-8925-df616bc01597 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:21:18.469558 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-1dab280b-db18-4bb5-8925-df616bc01597 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:21:18.473190 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1241/2479] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Jul 25 12:21:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:21:18.478011 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1239/2480] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:21:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 12:21:18.481309 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6266951-47ae-44a9-b74c-c8ec93037086 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:21:18.482400 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dab280b-db18-4bb5-8925-df616bc01597 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:21:18.488040 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6266951-47ae-44a9-b74c-c8ec93037086 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:21:18.491257 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dab280b-db18-4bb5-8925-df616bc01597 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:21:18.493346 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47713fab-e622-4204-8a2b-74fe27c5d040 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:21:18.493989 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47713fab-e622-4204-8a2b-74fe27c5d040 admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:18.494281 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47713fab-e622-4204-8a2b-74fe27c5d040 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:18.494565 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47713fab-e622-4204-8a2b-74fe27c5d040 admin admin] PATH_INFO: `/v3/projects` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:18.495289 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47713fab-e622-4204-8a2b-74fe27c5d040 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:21:18.496478 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e776bd1-088c-470e-9683-4659b05171d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:21:18.497115 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e776bd1-088c-470e-9683-4659b05171d0 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:18.497220 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47713fab-e622-4204-8a2b-74fe27c5d040 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:21:18.497405 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e776bd1-088c-470e-9683-4659b05171d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:18.497684 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e776bd1-088c-470e-9683-4659b05171d0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:18.498404 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e776bd1-088c-470e-9683-4659b05171d0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:21:18.498714 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e776bd1-088c-470e-9683-4659b05171d0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:21:18.500432 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e776bd1-088c-470e-9683-4659b05171d0 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:21:18.514242 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1240/2481] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Jul 25 12:21:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 12:21:18.518834 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e776bd1-088c-470e-9683-4659b05171d0 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:21:18.529910 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e776bd1-088c-470e-9683-4659b05171d0 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:21:18.531397 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1242/2482] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Jul 25 12:21:18 2022] POST /identity/v3/projects => generated 401 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:21:18.535097 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47713fab-e622-4204-8a2b-74fe27c5d040 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:21:18.584900 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:21:18.591335 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ee2c55-dcbc-4585-8390-26cc8d5de4c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:21:18.592036 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ee2c55-dcbc-4585-8390-26cc8d5de4c7 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:18.592309 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ee2c55-dcbc-4585-8390-26cc8d5de4c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:18.592615 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ee2c55-dcbc-4585-8390-26cc8d5de4c7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:18.593663 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ee2c55-dcbc-4585-8390-26cc8d5de4c7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:21:18.595196 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ee2c55-dcbc-4585-8390-26cc8d5de4c7 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:21:18.596943 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47713fab-e622-4204-8a2b-74fe27c5d040 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:21:18.602236 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152c223f-27af-4128-ac7c-bfa7bef47a30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:21:18.602938 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152c223f-27af-4128-ac7c-bfa7bef47a30 admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:18.603233 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152c223f-27af-4128-ac7c-bfa7bef47a30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:18.603518 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152c223f-27af-4128-ac7c-bfa7bef47a30 admin admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:18.604254 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152c223f-27af-4128-ac7c-bfa7bef47a30 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:21:18.606250 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152c223f-27af-4128-ac7c-bfa7bef47a30 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:21:18.623239 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1241/2483] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Jul 25 12:21:18 2022] POST /identity/v3/projects => generated 367 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 12:21:18.627379 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ee2c55-dcbc-4585-8390-26cc8d5de4c7 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:21:18.631350 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1243/2484] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Jul 25 12:21:18 2022] POST /identity/v3/users => generated 301 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 12:21:18.632553 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152c223f-27af-4128-ac7c-bfa7bef47a30 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:21:18.643499 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152c223f-27af-4128-ac7c-bfa7bef47a30 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:21:18.692474 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ee2c55-dcbc-4585-8390-26cc8d5de4c7 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:21:18.697672 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3553c17-1b3d-4d6d-93d5-eaa585479acf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:21:18.698340 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3553c17-1b3d-4d6d-93d5-eaa585479acf admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:18.698627 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3553c17-1b3d-4d6d-93d5-eaa585479acf admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:18.698909 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3553c17-1b3d-4d6d-93d5-eaa585479acf admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:18.699019 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a720a5-5960-425f-877d-6a52079c0ff8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:21:18.699626 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3553c17-1b3d-4d6d-93d5-eaa585479acf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:21:18.699743 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a720a5-5960-425f-877d-6a52079c0ff8 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:18.699976 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a720a5-5960-425f-877d-6a52079c0ff8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:18.700259 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a720a5-5960-425f-877d-6a52079c0ff8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:18.700985 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a720a5-5960-425f-877d-6a52079c0ff8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:21:18.701556 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3553c17-1b3d-4d6d-93d5-eaa585479acf admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:21:18.702568 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a720a5-5960-425f-877d-6a52079c0ff8 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:21:18.715335 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1244/2485] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Jul 25 12:21:18 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 12:21:18.716525 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a720a5-5960-425f-877d-6a52079c0ff8 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:21:18.722683 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1242/2486] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Jul 25 12:21:18 2022] POST /identity/v3/users => generated 284 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 12:21:18.726784 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3553c17-1b3d-4d6d-93d5-eaa585479acf admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:21:18.727008 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a720a5-5960-425f-877d-6a52079c0ff8 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:21:18.731925 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9575bad2-0004-471a-806c-2f7a16bd0f8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:21:18.732585 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9575bad2-0004-471a-806c-2f7a16bd0f8d admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:18.732879 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9575bad2-0004-471a-806c-2f7a16bd0f8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:18.733165 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9575bad2-0004-471a-806c-2f7a16bd0f8d admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:18.733893 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9575bad2-0004-471a-806c-2f7a16bd0f8d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:21:18.735511 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9575bad2-0004-471a-806c-2f7a16bd0f8d admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:21:18.736340 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3553c17-1b3d-4d6d-93d5-eaa585479acf admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:21:18.740889 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc6dd16-4702-4819-bd3b-93263b24610c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:21:18.741517 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc6dd16-4702-4819-bd3b-93263b24610c admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:18.741796 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc6dd16-4702-4819-bd3b-93263b24610c admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:18.742591 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc6dd16-4702-4819-bd3b-93263b24610c admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:18.743346 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc6dd16-4702-4819-bd3b-93263b24610c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:21:18.744973 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc6dd16-4702-4819-bd3b-93263b24610c admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:21:18.747813 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1245/2487] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Jul 25 12:21:18 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:21:18.751874 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9575bad2-0004-471a-806c-2f7a16bd0f8d admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:21:18.761898 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1243/2488] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Jul 25 12:21:18 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:21:18.762267 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9575bad2-0004-471a-806c-2f7a16bd0f8d admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:21:18.763128 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc6dd16-4702-4819-bd3b-93263b24610c admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:21:18.766997 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b181ca3-4b47-484b-a738-41fc5ac79cd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:21:18.767894 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b181ca3-4b47-484b-a738-41fc5ac79cd5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:18.768188 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b181ca3-4b47-484b-a738-41fc5ac79cd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:18.768468 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b181ca3-4b47-484b-a738-41fc5ac79cd5 admin admin] PATH_INFO: `/v3/projects/dc9ab4a4cde647868f69471832dc70a5/users/c713400ef9604efd91d62197c5fb951c/roles/472313ae69174578824df0bed6390744` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:18.773061 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc6dd16-4702-4819-bd3b-93263b24610c admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:21:18.780634 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891811a4-6ed5-4084-914f-0750402ee446 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:21:18.781306 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891811a4-6ed5-4084-914f-0750402ee446 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:18.781716 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891811a4-6ed5-4084-914f-0750402ee446 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:18.781997 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891811a4-6ed5-4084-914f-0750402ee446 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:18.782598 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891811a4-6ed5-4084-914f-0750402ee446 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:21:18.784267 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891811a4-6ed5-4084-914f-0750402ee446 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:21:18.787223 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b181ca3-4b47-484b-a738-41fc5ac79cd5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc9ab4a4cde647868f69471832dc70a5, user_id=c713400ef9604efd91d62197c5fb951c, role_id=472313ae69174578824df0bed6390744)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:21:18.790522 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b181ca3-4b47-484b-a738-41fc5ac79cd5 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:21:18.795492 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1244/2489] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Jul 25 12:21:18 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:21:18.799771 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891811a4-6ed5-4084-914f-0750402ee446 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:21:18.804372 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1246/2490] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Jul 25 12:21:18 2022] PUT /identity/v3/projects/dc9ab4a4cde647868f69471832dc70a5/users/c713400ef9604efd91d62197c5fb951c/roles/472313ae69174578824df0bed6390744 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 12:21:18.811207 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0552b091-7b64-4380-815c-e557608cd5cb None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:18.811507 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0552b091-7b64-4380-815c-e557608cd5cb None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:18.811821 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0552b091-7b64-4380-815c-e557608cd5cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:18.813579 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891811a4-6ed5-4084-914f-0750402ee446 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:21:18.830204 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-0552b091-7b64-4380-815c-e557608cd5cb None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:21:18.831432 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-0552b091-7b64-4380-815c-e557608cd5cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:21:18.838863 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:21:18.871612 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58353bb-cbd0-41d4-a888-e84e33ff36fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:21:18.872527 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58353bb-cbd0-41d4-a888-e84e33ff36fc admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:18.872933 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58353bb-cbd0-41d4-a888-e84e33ff36fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:18.873249 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58353bb-cbd0-41d4-a888-e84e33ff36fc admin admin] PATH_INFO: `/v3/projects/39471a6491054b2fb2e3f126159fdede/users/b507f83f1f6d406ba9a686486a8353a6/roles/472313ae69174578824df0bed6390744` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:18.887938 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58353bb-cbd0-41d4-a888-e84e33ff36fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=39471a6491054b2fb2e3f126159fdede, user_id=b507f83f1f6d406ba9a686486a8353a6, role_id=472313ae69174578824df0bed6390744)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:21:18.890583 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58353bb-cbd0-41d4-a888-e84e33ff36fc admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:21:18.891852 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1247/2491] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:21:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 12:21:18.902489 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0552b091-7b64-4380-815c-e557608cd5cb None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:21:18.903988 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-0552b091-7b64-4380-815c-e557608cd5cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:21:18.904525 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1245/2492] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Jul 25 12:21:18 2022] PUT /identity/v3/projects/39471a6491054b2fb2e3f126159fdede/users/b507f83f1f6d406ba9a686486a8353a6/roles/472313ae69174578824df0bed6390744 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 12:21:18.916307 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4716704f-961b-4d5a-894b-f0405000485c None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:18.916603 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4716704f-961b-4d5a-894b-f0405000485c None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:18.916880 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4716704f-961b-4d5a-894b-f0405000485c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:18.943980 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-4716704f-961b-4d5a-894b-f0405000485c None None] MFA Rules not processed for user `b507f83f1f6d406ba9a686486a8353a6`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:21:18.945248 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-4716704f-961b-4d5a-894b-f0405000485c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:21:18.949771 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:21:18.962848 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0552b091-7b64-4380-815c-e557608cd5cb None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:21:19.010995 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1246/2493] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:21:18 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 12:21:19.015369 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4716704f-961b-4d5a-894b-f0405000485c None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:21:19.015536 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe93bfd-aac2-4b88-a52b-c7bcf80ac4e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51733781201243e8abf9eca39b207750', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:21:19.016112 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe93bfd-aac2-4b88-a52b-c7bcf80ac4e5 service neutron] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:19.016400 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe93bfd-aac2-4b88-a52b-c7bcf80ac4e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:19.016682 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe93bfd-aac2-4b88-a52b-c7bcf80ac4e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:19.087308 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe93bfd-aac2-4b88-a52b-c7bcf80ac4e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:21:19.089262 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe93bfd-aac2-4b88-a52b-c7bcf80ac4e5 service neutron] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:21:19.100577 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1248/2494] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:21:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 12:21:19.102271 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4716704f-961b-4d5a-894b-f0405000485c None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:21:19.106964 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e7fe54-e8a5-4258-9aad-2bd966e3a87e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:21:19.107663 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e7fe54-e8a5-4258-9aad-2bd966e3a87e admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:19.107986 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e7fe54-e8a5-4258-9aad-2bd966e3a87e admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:19.108273 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e7fe54-e8a5-4258-9aad-2bd966e3a87e admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:19.108985 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e7fe54-e8a5-4258-9aad-2bd966e3a87e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:21:19.110933 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e7fe54-e8a5-4258-9aad-2bd966e3a87e admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:21:19.139406 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1247/2495] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Jul 25 12:21:19 2022] POST /identity/v3/projects => generated 367 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 12:21:19.143403 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe93bfd-aac2-4b88-a52b-c7bcf80ac4e5 service neutron] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:21:19.153874 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe93bfd-aac2-4b88-a52b-c7bcf80ac4e5 service neutron] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:21:19.214853 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d4ee75-f63f-4fdd-9675-033eff73ff1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:21:19.215531 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d4ee75-f63f-4fdd-9675-033eff73ff1b admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:19.215849 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d4ee75-f63f-4fdd-9675-033eff73ff1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:19.216134 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d4ee75-f63f-4fdd-9675-033eff73ff1b admin admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:19.216844 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d4ee75-f63f-4fdd-9675-033eff73ff1b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:21:19.218993 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d4ee75-f63f-4fdd-9675-033eff73ff1b admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:21:19.245465 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1249/2496] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Jul 25 12:21:19 2022] POST /identity/v3/users => generated 290 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:21:19.247086 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e7fe54-e8a5-4258-9aad-2bd966e3a87e admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:21:19.258460 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e7fe54-e8a5-4258-9aad-2bd966e3a87e admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:21:19.263882 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6a6153-50eb-4e8c-b4ff-1fd61202697f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:21:19.264715 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6a6153-50eb-4e8c-b4ff-1fd61202697f admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:19.265011 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6a6153-50eb-4e8c-b4ff-1fd61202697f admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:19.265758 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6a6153-50eb-4e8c-b4ff-1fd61202697f admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:19.266037 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6a6153-50eb-4e8c-b4ff-1fd61202697f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:21:19.267693 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6a6153-50eb-4e8c-b4ff-1fd61202697f admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:21:19.280928 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1248/2497] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Jul 25 12:21:19 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 12:21:19.285169 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d4ee75-f63f-4fdd-9675-033eff73ff1b admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:21:19.296160 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d4ee75-f63f-4fdd-9675-033eff73ff1b admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:21:19.300830 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf13bd35-6398-4308-84fd-cb0e7bbfd657 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:21:19.301807 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf13bd35-6398-4308-84fd-cb0e7bbfd657 admin admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:19.302166 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf13bd35-6398-4308-84fd-cb0e7bbfd657 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:19.302464 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf13bd35-6398-4308-84fd-cb0e7bbfd657 admin admin] PATH_INFO: `/v3/projects/9cc67c4674b64223ae2a8c9e7de9b6d4/users/80b393d9534b4cb182c2b9ad5bed4f83/roles/a7d20885b6d94928a34b3cc5b3a00006` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:19.326046 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf13bd35-6398-4308-84fd-cb0e7bbfd657 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9cc67c4674b64223ae2a8c9e7de9b6d4, user_id=80b393d9534b4cb182c2b9ad5bed4f83, role_id=a7d20885b6d94928a34b3cc5b3a00006)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:21:19.328771 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf13bd35-6398-4308-84fd-cb0e7bbfd657 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:21:19.343088 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1250/2498] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Jul 25 12:21:19 2022] PUT /identity/v3/projects/9cc67c4674b64223ae2a8c9e7de9b6d4/users/80b393d9534b4cb182c2b9ad5bed4f83/roles/a7d20885b6d94928a34b3cc5b3a00006 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:21:19.347389 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6a6153-50eb-4e8c-b4ff-1fd61202697f admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:21:19.359409 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6a6153-50eb-4e8c-b4ff-1fd61202697f admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:21:19.372813 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:21:19.439001 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b209ba1-7d3b-4a30-a331-7d47b8903f11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:21:19.439869 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b209ba1-7d3b-4a30-a331-7d47b8903f11 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:19.440265 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b209ba1-7d3b-4a30-a331-7d47b8903f11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:19.440669 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b209ba1-7d3b-4a30-a331-7d47b8903f11 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:19.441688 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b209ba1-7d3b-4a30-a331-7d47b8903f11 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:21:19.443537 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b209ba1-7d3b-4a30-a331-7d47b8903f11 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:21:19.463593 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1249/2499] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Jul 25 12:21:19 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 12:21:19.464989 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf13bd35-6398-4308-84fd-cb0e7bbfd657 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:21:19.476220 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf13bd35-6398-4308-84fd-cb0e7bbfd657 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:21:19.481881 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c943e9-7e01-41ef-8a0e-8bcd627cf623 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:21:19.482854 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c943e9-7e01-41ef-8a0e-8bcd627cf623 admin admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:19.483216 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c943e9-7e01-41ef-8a0e-8bcd627cf623 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:19.483621 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c943e9-7e01-41ef-8a0e-8bcd627cf623 admin admin] PATH_INFO: `/v3/projects/9cc67c4674b64223ae2a8c9e7de9b6d4/users/80b393d9534b4cb182c2b9ad5bed4f83/roles/a7d20885b6d94928a34b3cc5b3a00006` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:19.486122 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c943e9-7e01-41ef-8a0e-8bcd627cf623 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9cc67c4674b64223ae2a8c9e7de9b6d4, user_id=80b393d9534b4cb182c2b9ad5bed4f83, role_id=a7d20885b6d94928a34b3cc5b3a00006)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:21:19.489150 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c943e9-7e01-41ef-8a0e-8bcd627cf623 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:21:19.497178 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:21:19.502135 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1251/2500] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Jul 25 12:21:19 2022] PUT /identity/v3/projects/9cc67c4674b64223ae2a8c9e7de9b6d4/users/80b393d9534b4cb182c2b9ad5bed4f83/roles/a7d20885b6d94928a34b3cc5b3a00006 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:21:19.514109 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281f721d-a83a-40ae-8f55-1896ba70364a None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:19.514408 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281f721d-a83a-40ae-8f55-1896ba70364a None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:19.514700 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281f721d-a83a-40ae-8f55-1896ba70364a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:19.544553 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-281f721d-a83a-40ae-8f55-1896ba70364a None None] MFA Rules not processed for user `80b393d9534b4cb182c2b9ad5bed4f83`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:21:19.545841 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-281f721d-a83a-40ae-8f55-1896ba70364a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:21:19.615710 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1250/2501] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:21:19 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:21:19.627656 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c943e9-7e01-41ef-8a0e-8bcd627cf623 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:21:19.629395 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-51c943e9-7e01-41ef-8a0e-8bcd627cf623 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:21:19.690555 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c943e9-7e01-41ef-8a0e-8bcd627cf623 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:21:19.694367 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfa1cc0-9c6c-4bd5-830f-1aaa7ed7c1a7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f89ce6f0cf04373a414d927fa0f8a7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:21:19.695037 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfa1cc0-9c6c-4bd5-830f-1aaa7ed7c1a7 service cinder] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:19.695878 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfa1cc0-9c6c-4bd5-830f-1aaa7ed7c1a7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:19.696159 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfa1cc0-9c6c-4bd5-830f-1aaa7ed7c1a7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:19.705997 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfa1cc0-9c6c-4bd5-830f-1aaa7ed7c1a7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:21:19.707460 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfa1cc0-9c6c-4bd5-830f-1aaa7ed7c1a7 service cinder] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:21:19.719537 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1252/2502] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:21:19 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:21:19.761429 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e2ac2a-0283-45d3-a657-4c424dea7061 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:19.761728 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e2ac2a-0283-45d3-a657-4c424dea7061 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:19.762104 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e2ac2a-0283-45d3-a657-4c424dea7061 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:19.810197 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-45e2ac2a-0283-45d3-a657-4c424dea7061 None None] MFA Rules not processed for user `c713400ef9604efd91d62197c5fb951c`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:21:19.811493 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-45e2ac2a-0283-45d3-a657-4c424dea7061 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:21:19.813759 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfa1cc0-9c6c-4bd5-830f-1aaa7ed7c1a7 service cinder] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:21:19.826061 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfa1cc0-9c6c-4bd5-830f-1aaa7ed7c1a7 service cinder] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:21:19.830607 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e6b7ed-2947-4ac0-8e10-c3f76f967083 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f89ce6f0cf04373a414d927fa0f8a7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:21:19.831341 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e6b7ed-2947-4ac0-8e10-c3f76f967083 service cinder] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:19.831741 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e6b7ed-2947-4ac0-8e10-c3f76f967083 service cinder] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:19.832100 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e6b7ed-2947-4ac0-8e10-c3f76f967083 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:19.890371 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1251/2503] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:21:19 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:21:19.894759 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e2ac2a-0283-45d3-a657-4c424dea7061 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:21:19.904060 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e6b7ed-2947-4ac0-8e10-c3f76f967083 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:21:19.905555 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e6b7ed-2947-4ac0-8e10-c3f76f967083 service cinder] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:21:19.918433 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1253/2504] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:21:19 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 12:21:19.926627 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:21:19.967776 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e2ac2a-0283-45d3-a657-4c424dea7061 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:21:19.972721 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25927780-67ce-4860-9522-f5d974c74419 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:21:19.973330 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25927780-67ce-4860-9522-f5d974c74419 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:19.973628 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25927780-67ce-4860-9522-f5d974c74419 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:19.973976 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25927780-67ce-4860-9522-f5d974c74419 admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:19.974731 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25927780-67ce-4860-9522-f5d974c74419 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:21:19.977340 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25927780-67ce-4860-9522-f5d974c74419 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:21:20.004903 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1252/2505] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Jul 25 12:21:19 2022] POST /identity/v3/projects => generated 403 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 12:21:20.009273 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e6b7ed-2947-4ac0-8e10-c3f76f967083 service cinder] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:21:20.020615 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e6b7ed-2947-4ac0-8e10-c3f76f967083 service cinder] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:21:20.082833 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e73044a-421d-4451-95bb-a903eb6708be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:21:20.083585 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e73044a-421d-4451-95bb-a903eb6708be admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:20.083908 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e73044a-421d-4451-95bb-a903eb6708be admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:20.084244 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e73044a-421d-4451-95bb-a903eb6708be admin admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:20.085160 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e73044a-421d-4451-95bb-a903eb6708be admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:21:20.087536 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e73044a-421d-4451-95bb-a903eb6708be admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:21:20.109520 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1254/2506] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Jul 25 12:21:20 2022] POST /identity/v3/users => generated 308 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:21:20.113756 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25927780-67ce-4860-9522-f5d974c74419 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:21:20.125329 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25927780-67ce-4860-9522-f5d974c74419 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:21:20.130146 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f184b374-f9eb-4035-aa73-3189b02c9aa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:21:20.130809 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f184b374-f9eb-4035-aa73-3189b02c9aa9 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:20.131096 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f184b374-f9eb-4035-aa73-3189b02c9aa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:20.131379 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f184b374-f9eb-4035-aa73-3189b02c9aa9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:20.132144 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f184b374-f9eb-4035-aa73-3189b02c9aa9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:21:20.134388 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f184b374-f9eb-4035-aa73-3189b02c9aa9 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:21:20.147708 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1253/2507] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Jul 25 12:21:20 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:21:20.152333 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e73044a-421d-4451-95bb-a903eb6708be admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:21:20.163018 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e73044a-421d-4451-95bb-a903eb6708be admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:21:20.168688 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc29008-9fe8-48d0-beb9-fa03d90b6632 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:21:20.169585 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc29008-9fe8-48d0-beb9-fa03d90b6632 admin admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:20.169904 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc29008-9fe8-48d0-beb9-fa03d90b6632 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:20.170154 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc29008-9fe8-48d0-beb9-fa03d90b6632 admin admin] PATH_INFO: `/v3/projects/1d07a9d436174c2f8b14273498d64537/users/c51273ac1e3a44ec813afe229292556d/roles/a7d20885b6d94928a34b3cc5b3a00006` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:20.187182 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc29008-9fe8-48d0-beb9-fa03d90b6632 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d07a9d436174c2f8b14273498d64537, user_id=c51273ac1e3a44ec813afe229292556d, role_id=a7d20885b6d94928a34b3cc5b3a00006)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:21:20.190168 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc29008-9fe8-48d0-beb9-fa03d90b6632 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:21:20.206032 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1255/2508] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Jul 25 12:21:20 2022] PUT /identity/v3/projects/1d07a9d436174c2f8b14273498d64537/users/c51273ac1e3a44ec813afe229292556d/roles/a7d20885b6d94928a34b3cc5b3a00006 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:21:20.210042 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f184b374-f9eb-4035-aa73-3189b02c9aa9 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:21:20.221215 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f184b374-f9eb-4035-aa73-3189b02c9aa9 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:21:20.283448 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb8a31a-e566-4037-8c4f-4d7e1a427a0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:21:20.284175 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb8a31a-e566-4037-8c4f-4d7e1a427a0b admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:20.284465 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb8a31a-e566-4037-8c4f-4d7e1a427a0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:20.284747 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb8a31a-e566-4037-8c4f-4d7e1a427a0b admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:20.285534 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb8a31a-e566-4037-8c4f-4d7e1a427a0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:21:20.287613 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb8a31a-e566-4037-8c4f-4d7e1a427a0b admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:21:20.300420 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1254/2509] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Jul 25 12:21:20 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:21:20.305488 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc29008-9fe8-48d0-beb9-fa03d90b6632 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:21:20.316155 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc29008-9fe8-48d0-beb9-fa03d90b6632 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:21:20.320923 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d759b15f-0871-4358-bf61-d47eefe6aff2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:21:20.321799 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d759b15f-0871-4358-bf61-d47eefe6aff2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:20.322091 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d759b15f-0871-4358-bf61-d47eefe6aff2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:20.322376 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d759b15f-0871-4358-bf61-d47eefe6aff2 admin admin] PATH_INFO: `/v3/projects/1d07a9d436174c2f8b14273498d64537/users/c51273ac1e3a44ec813afe229292556d/roles/a7d20885b6d94928a34b3cc5b3a00006` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:20.324414 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d759b15f-0871-4358-bf61-d47eefe6aff2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d07a9d436174c2f8b14273498d64537, user_id=c51273ac1e3a44ec813afe229292556d, role_id=a7d20885b6d94928a34b3cc5b3a00006)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:21:20.327044 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d759b15f-0871-4358-bf61-d47eefe6aff2 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:21:20.329332 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:21:20.338982 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1256/2510] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Jul 25 12:21:20 2022] PUT /identity/v3/projects/1d07a9d436174c2f8b14273498d64537/users/c51273ac1e3a44ec813afe229292556d/roles/a7d20885b6d94928a34b3cc5b3a00006 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:21:20.968436 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1084f31a-be50-4230-a557-6c6ff40daa9f None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:20.968921 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1084f31a-be50-4230-a557-6c6ff40daa9f None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:20.969392 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1084f31a-be50-4230-a557-6c6ff40daa9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:21.000659 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-1084f31a-be50-4230-a557-6c6ff40daa9f None None] MFA Rules not processed for user `c51273ac1e3a44ec813afe229292556d`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:21:21.001915 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-1084f31a-be50-4230-a557-6c6ff40daa9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:21:21.073544 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1255/2511] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:21:20 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:21:22.338786 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d759b15f-0871-4358-bf61-d47eefe6aff2 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:21:22.340618 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-d759b15f-0871-4358-bf61-d47eefe6aff2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:21:22.392024 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1084f31a-be50-4230-a557-6c6ff40daa9f None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:21:22.418728 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:21:22.421296 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d759b15f-0871-4358-bf61-d47eefe6aff2 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:21:22.426896 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f653155-9464-45d7-9025-d76f005b9b06 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8012afab6cac4358ab878732e58d0689', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:21:22.427621 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f653155-9464-45d7-9025-d76f005b9b06 service placement] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:22.427870 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f653155-9464-45d7-9025-d76f005b9b06 service placement] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:22.428156 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f653155-9464-45d7-9025-d76f005b9b06 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:22.430006 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-2f653155-9464-45d7-9025-d76f005b9b06 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:21:22.442306 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f653155-9464-45d7-9025-d76f005b9b06 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:21:22.444020 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f653155-9464-45d7-9025-d76f005b9b06 service placement] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:21:22.454644 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1257/2512] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:21:22 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:21:22.459745 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1084f31a-be50-4230-a557-6c6ff40daa9f None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:21:22.464971 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f636b5-3222-47bc-9957-2704ecd09735 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:21:22.465883 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f636b5-3222-47bc-9957-2704ecd09735 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:22.466177 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f636b5-3222-47bc-9957-2704ecd09735 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:22.466464 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f636b5-3222-47bc-9957-2704ecd09735 admin admin] PATH_INFO: `/v3/users/c713400ef9604efd91d62197c5fb951c` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:22.467712 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f636b5-3222-47bc-9957-2704ecd09735 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c713400ef9604efd91d62197c5fb951c)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:21:22.469657 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f636b5-3222-47bc-9957-2704ecd09735 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:21:22.522862 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-b4f636b5-3222-47bc-9957-2704ecd09735 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c713400ef9604efd91d62197c5fb951c'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:22.530522 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-b4f636b5-3222-47bc-9957-2704ecd09735 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c713400ef9604efd91d62197c5fb951c'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:22.537543 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-b4f636b5-3222-47bc-9957-2704ecd09735 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c713400ef9604efd91d62197c5fb951c'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:22.550173 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-b4f636b5-3222-47bc-9957-2704ecd09735 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c713400ef9604efd91d62197c5fb951c'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:22.550861 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-b4f636b5-3222-47bc-9957-2704ecd09735 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c713400ef9604efd91d62197c5fb951c'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:22.556975 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1256/2513] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Jul 25 12:21:22 2022] DELETE /identity/v3/users/c713400ef9604efd91d62197c5fb951c => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 12:21:22.742293 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f653155-9464-45d7-9025-d76f005b9b06 service placement] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:21:22.753255 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f653155-9464-45d7-9025-d76f005b9b06 service placement] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:21:22.824108 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e227b8-588a-4173-af3c-21a66f899a3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:21:22.825189 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e227b8-588a-4173-af3c-21a66f899a3e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:22.825491 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e227b8-588a-4173-af3c-21a66f899a3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:22.825900 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e227b8-588a-4173-af3c-21a66f899a3e admin admin] PATH_INFO: `/v3/projects/dc9ab4a4cde647868f69471832dc70a5` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:22.827122 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e227b8-588a-4173-af3c-21a66f899a3e admin admin] RBAC: Authorizing `identity:delete_project(project_id=dc9ab4a4cde647868f69471832dc70a5)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:21:22.829206 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e227b8-588a-4173-af3c-21a66f899a3e admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:21:22.883738 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-a7e227b8-588a-4173-af3c-21a66f899a3e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dc9ab4a4cde647868f69471832dc70a5', 'request_id': 'req-a7e227b8-588a-4173-af3c-21a66f899a3e'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:22.892782 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-a7e227b8-588a-4173-af3c-21a66f899a3e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dc9ab4a4cde647868f69471832dc70a5', 'request_id': 'req-a7e227b8-588a-4173-af3c-21a66f899a3e'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:22.899814 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-a7e227b8-588a-4173-af3c-21a66f899a3e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dc9ab4a4cde647868f69471832dc70a5', 'request_id': 'req-a7e227b8-588a-4173-af3c-21a66f899a3e'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:22.911480 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-a7e227b8-588a-4173-af3c-21a66f899a3e admin admin] The token cache is being invalidate because project dc9ab4a4cde647868f69471832dc70a5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80298) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 12:21:22.911800 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-a7e227b8-588a-4173-af3c-21a66f899a3e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:22.912452 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-a7e227b8-588a-4173-af3c-21a66f899a3e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:22.916926 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1258/2514] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Jul 25 12:21:22 2022] DELETE /identity/v3/projects/dc9ab4a4cde647868f69471832dc70a5 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:21:22.918589 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f636b5-3222-47bc-9957-2704ecd09735 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:21:22.920206 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-b4f636b5-3222-47bc-9957-2704ecd09735 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:21:22.936494 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f636b5-3222-47bc-9957-2704ecd09735 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:21:22.999737 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a814c6b6-e172-4a81-adf7-b2fac4a4efd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:21:23.000635 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a814c6b6-e172-4a81-adf7-b2fac4a4efd3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:23.000923 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a814c6b6-e172-4a81-adf7-b2fac4a4efd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:23.001209 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a814c6b6-e172-4a81-adf7-b2fac4a4efd3 admin admin] PATH_INFO: `/v3/users/c51273ac1e3a44ec813afe229292556d` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:23.002461 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a814c6b6-e172-4a81-adf7-b2fac4a4efd3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c51273ac1e3a44ec813afe229292556d)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:21:23.004437 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a814c6b6-e172-4a81-adf7-b2fac4a4efd3 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:21:23.055897 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:21:23.058489 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-a814c6b6-e172-4a81-adf7-b2fac4a4efd3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c51273ac1e3a44ec813afe229292556d'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:23.067146 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-a814c6b6-e172-4a81-adf7-b2fac4a4efd3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c51273ac1e3a44ec813afe229292556d'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:23.082082 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-a814c6b6-e172-4a81-adf7-b2fac4a4efd3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c51273ac1e3a44ec813afe229292556d'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:23.095426 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-a814c6b6-e172-4a81-adf7-b2fac4a4efd3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c51273ac1e3a44ec813afe229292556d'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:23.096264 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-a814c6b6-e172-4a81-adf7-b2fac4a4efd3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c51273ac1e3a44ec813afe229292556d'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:23.103383 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1257/2515] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Jul 25 12:21:22 2022] DELETE /identity/v3/users/c51273ac1e3a44ec813afe229292556d => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:21:23.278558 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e227b8-588a-4173-af3c-21a66f899a3e admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:21:23.280048 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-a7e227b8-588a-4173-af3c-21a66f899a3e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:21:23.298452 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e227b8-588a-4173-af3c-21a66f899a3e admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:21:23.308668 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:INSERT=2,keystone:SELECT=91,keystone:DELETE=7 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:21:23.365941 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60775cc-fd5e-4858-ac1e-7e0749419d90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:21:23.367010 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60775cc-fd5e-4858-ac1e-7e0749419d90 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:23.367299 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60775cc-fd5e-4858-ac1e-7e0749419d90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:23.367657 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60775cc-fd5e-4858-ac1e-7e0749419d90 admin admin] PATH_INFO: `/v3/projects/1d07a9d436174c2f8b14273498d64537` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:23.368859 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60775cc-fd5e-4858-ac1e-7e0749419d90 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1d07a9d436174c2f8b14273498d64537)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:21:23.370810 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60775cc-fd5e-4858-ac1e-7e0749419d90 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:21:23.428152 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-f60775cc-fd5e-4858-ac1e-7e0749419d90 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1d07a9d436174c2f8b14273498d64537', 'request_id': 'req-f60775cc-fd5e-4858-ac1e-7e0749419d90'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:23.439830 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-f60775cc-fd5e-4858-ac1e-7e0749419d90 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1d07a9d436174c2f8b14273498d64537', 'request_id': 'req-f60775cc-fd5e-4858-ac1e-7e0749419d90'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:23.446344 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-f60775cc-fd5e-4858-ac1e-7e0749419d90 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1d07a9d436174c2f8b14273498d64537', 'request_id': 'req-f60775cc-fd5e-4858-ac1e-7e0749419d90'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:23.457782 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-f60775cc-fd5e-4858-ac1e-7e0749419d90 admin admin] The token cache is being invalidate because project 1d07a9d436174c2f8b14273498d64537 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80298) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 12:21:23.457974 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-f60775cc-fd5e-4858-ac1e-7e0749419d90 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:23.458652 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-f60775cc-fd5e-4858-ac1e-7e0749419d90 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:23.460269 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1259/2516] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Jul 25 12:21:23 2022] DELETE /identity/v3/projects/1d07a9d436174c2f8b14273498d64537 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:21:23.466774 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d3f0e5-e129-4716-9307-88ded9feff71 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:23.467251 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d3f0e5-e129-4716-9307-88ded9feff71 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:23.467366 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d3f0e5-e129-4716-9307-88ded9feff71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:23.488370 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-19d3f0e5-e129-4716-9307-88ded9feff71 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:21:23.489591 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-19d3f0e5-e129-4716-9307-88ded9feff71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:21:23.553192 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1258/2517] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:21:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:21:23.562405 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b5adc2-081d-4294-b6cd-6f03f148b3aa None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:23.562810 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b5adc2-081d-4294-b6cd-6f03f148b3aa None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:23.563157 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b5adc2-081d-4294-b6cd-6f03f148b3aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:23.583484 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-76b5adc2-081d-4294-b6cd-6f03f148b3aa None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:21:23.584780 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-76b5adc2-081d-4294-b6cd-6f03f148b3aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:21:23.593594 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1260/2518] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:21:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:21:23.597931 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d3f0e5-e129-4716-9307-88ded9feff71 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:21:23.607994 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d3f0e5-e129-4716-9307-88ded9feff71 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:21:23.612488 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5cd69a-e588-4fcc-bf99-a5c56ce10065 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:21:23.613136 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5cd69a-e588-4fcc-bf99-a5c56ce10065 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:23.613422 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5cd69a-e588-4fcc-bf99-a5c56ce10065 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:23.613702 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5cd69a-e588-4fcc-bf99-a5c56ce10065 admin admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:23.614427 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5cd69a-e588-4fcc-bf99-a5c56ce10065 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:21:23.614734 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5cd69a-e588-4fcc-bf99-a5c56ce10065 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:21:23.616362 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5cd69a-e588-4fcc-bf99-a5c56ce10065 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:21:23.628155 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1259/2519] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Jul 25 12:21:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:21:23.633305 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b5adc2-081d-4294-b6cd-6f03f148b3aa None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:21:23.643670 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b5adc2-081d-4294-b6cd-6f03f148b3aa None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:21:23.648346 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd4d8a6-db39-4bec-b521-1cca3f659780 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:21:23.648992 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd4d8a6-db39-4bec-b521-1cca3f659780 admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:23.649281 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd4d8a6-db39-4bec-b521-1cca3f659780 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:23.649816 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd4d8a6-db39-4bec-b521-1cca3f659780 admin admin] PATH_INFO: `/v3/projects` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:23.650538 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd4d8a6-db39-4bec-b521-1cca3f659780 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:21:23.652483 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd4d8a6-db39-4bec-b521-1cca3f659780 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:21:23.678854 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1261/2520] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Jul 25 12:21:23 2022] POST /identity/v3/projects => generated 381 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:21:23.683565 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5cd69a-e588-4fcc-bf99-a5c56ce10065 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:21:23.693443 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5cd69a-e588-4fcc-bf99-a5c56ce10065 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:21:23.748276 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4726ea6a-9e76-444d-af38-cb07b9643bd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:21:23.749036 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4726ea6a-9e76-444d-af38-cb07b9643bd3 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:23.749329 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4726ea6a-9e76-444d-af38-cb07b9643bd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:23.749812 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4726ea6a-9e76-444d-af38-cb07b9643bd3 admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:23.750519 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4726ea6a-9e76-444d-af38-cb07b9643bd3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:21:23.752512 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4726ea6a-9e76-444d-af38-cb07b9643bd3 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:21:23.786100 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1260/2521] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Jul 25 12:21:23 2022] POST /identity/v3/users => generated 291 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:21:23.789848 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd4d8a6-db39-4bec-b521-1cca3f659780 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:21:23.800327 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd4d8a6-db39-4bec-b521-1cca3f659780 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:21:23.805468 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c63db17-69a4-46fe-8d09-e480052bed1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:21:23.806137 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c63db17-69a4-46fe-8d09-e480052bed1a admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:23.806427 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c63db17-69a4-46fe-8d09-e480052bed1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:23.806711 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c63db17-69a4-46fe-8d09-e480052bed1a admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:23.807438 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c63db17-69a4-46fe-8d09-e480052bed1a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:21:23.809080 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c63db17-69a4-46fe-8d09-e480052bed1a admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:21:23.819021 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1262/2522] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Jul 25 12:21:23 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 12:21:23.823669 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4726ea6a-9e76-444d-af38-cb07b9643bd3 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:21:23.833847 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4726ea6a-9e76-444d-af38-cb07b9643bd3 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:21:23.838691 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1c4054-e4d1-4855-bfda-2048688e4771 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:21:23.839357 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1c4054-e4d1-4855-bfda-2048688e4771 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:23.839686 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1c4054-e4d1-4855-bfda-2048688e4771 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:23.839969 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1c4054-e4d1-4855-bfda-2048688e4771 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:23.840705 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1c4054-e4d1-4855-bfda-2048688e4771 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:21:23.842339 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1c4054-e4d1-4855-bfda-2048688e4771 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:21:23.853211 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1261/2523] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Jul 25 12:21:23 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:21:23.857202 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c63db17-69a4-46fe-8d09-e480052bed1a admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:21:23.867718 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c63db17-69a4-46fe-8d09-e480052bed1a admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:21:23.872299 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e0642b-512f-4e91-9c1b-da6978f5f29d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:21:23.873173 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e0642b-512f-4e91-9c1b-da6978f5f29d admin admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:23.873461 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e0642b-512f-4e91-9c1b-da6978f5f29d admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:23.873762 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e0642b-512f-4e91-9c1b-da6978f5f29d admin admin] PATH_INFO: `/v3/projects/82b2b9991d98449897da4a889733c9c7/users/98b073de38f14638a767b5b3400bb40c/roles/472313ae69174578824df0bed6390744` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:23.888725 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e0642b-512f-4e91-9c1b-da6978f5f29d admin admin] RBAC: Authorizing `identity:create_grant(project_id=82b2b9991d98449897da4a889733c9c7, user_id=98b073de38f14638a767b5b3400bb40c, role_id=472313ae69174578824df0bed6390744)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:21:23.891384 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e0642b-512f-4e91-9c1b-da6978f5f29d admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:21:23.904536 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1263/2524] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Jul 25 12:21:23 2022] PUT /identity/v3/projects/82b2b9991d98449897da4a889733c9c7/users/98b073de38f14638a767b5b3400bb40c/roles/472313ae69174578824df0bed6390744 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 12:21:23.914385 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11910f78-a5fc-4028-b6eb-d6b00f23982e None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:23.914683 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11910f78-a5fc-4028-b6eb-d6b00f23982e None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:23.914997 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11910f78-a5fc-4028-b6eb-d6b00f23982e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:23.942723 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-11910f78-a5fc-4028-b6eb-d6b00f23982e None None] MFA Rules not processed for user `98b073de38f14638a767b5b3400bb40c`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:21:23.943972 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-11910f78-a5fc-4028-b6eb-d6b00f23982e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:21:23.956825 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:21:24.011923 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1262/2525] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:21:23 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:21:24.016414 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e0642b-512f-4e91-9c1b-da6978f5f29d admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:21:24.060034 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:21:24.080437 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e0642b-512f-4e91-9c1b-da6978f5f29d admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:21:24.085043 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fdae69-d1d5-460b-a0e8-218f6924cb00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:21:24.085759 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fdae69-d1d5-460b-a0e8-218f6924cb00 admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:24.086045 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fdae69-d1d5-460b-a0e8-218f6924cb00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:24.086325 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fdae69-d1d5-460b-a0e8-218f6924cb00 admin admin] PATH_INFO: `/v3/projects` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:24.087021 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fdae69-d1d5-460b-a0e8-218f6924cb00 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:21:24.088981 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fdae69-d1d5-460b-a0e8-218f6924cb00 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:21:24.114960 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1264/2526] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Jul 25 12:21:24 2022] POST /identity/v3/projects => generated 381 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:21:24.119778 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11910f78-a5fc-4028-b6eb-d6b00f23982e None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:21:24.131045 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11910f78-a5fc-4028-b6eb-d6b00f23982e None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:21:24.193502 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0b1e33-fb8a-44e2-badc-fbe7c762fc28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:21:24.194178 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0b1e33-fb8a-44e2-badc-fbe7c762fc28 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:24.194471 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0b1e33-fb8a-44e2-badc-fbe7c762fc28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:24.194758 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0b1e33-fb8a-44e2-badc-fbe7c762fc28 admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:24.195463 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0b1e33-fb8a-44e2-badc-fbe7c762fc28 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:21:24.197432 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0b1e33-fb8a-44e2-badc-fbe7c762fc28 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:21:24.219585 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1263/2527] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Jul 25 12:21:24 2022] POST /identity/v3/users => generated 297 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:21:24.223515 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fdae69-d1d5-460b-a0e8-218f6924cb00 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:21:24.234508 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fdae69-d1d5-460b-a0e8-218f6924cb00 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:21:24.239082 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1932b13-e56d-4532-bd8f-04bebdeb9805 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:21:24.239773 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1932b13-e56d-4532-bd8f-04bebdeb9805 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:24.240060 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1932b13-e56d-4532-bd8f-04bebdeb9805 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:24.240346 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1932b13-e56d-4532-bd8f-04bebdeb9805 admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:24.241053 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1932b13-e56d-4532-bd8f-04bebdeb9805 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:21:24.249861 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1932b13-e56d-4532-bd8f-04bebdeb9805 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:21:24.263651 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1265/2528] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Jul 25 12:21:24 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:21:24.267855 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0b1e33-fb8a-44e2-badc-fbe7c762fc28 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:21:24.278866 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0b1e33-fb8a-44e2-badc-fbe7c762fc28 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:21:24.283574 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddbabea-8cc8-4528-9ae3-c764f83dd250 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:21:24.284461 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddbabea-8cc8-4528-9ae3-c764f83dd250 admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:24.284750 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddbabea-8cc8-4528-9ae3-c764f83dd250 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:24.285073 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddbabea-8cc8-4528-9ae3-c764f83dd250 admin admin] PATH_INFO: `/v3/projects/6a8b04af7f0842a6863b89832d04610b/users/2ce731482c3d44d29b479593e9e1ff06/roles/a7d20885b6d94928a34b3cc5b3a00006` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:24.301815 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ddbabea-8cc8-4528-9ae3-c764f83dd250 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6a8b04af7f0842a6863b89832d04610b, user_id=2ce731482c3d44d29b479593e9e1ff06, role_id=a7d20885b6d94928a34b3cc5b3a00006)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:21:24.305090 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ddbabea-8cc8-4528-9ae3-c764f83dd250 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:21:24.319229 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1264/2529] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Jul 25 12:21:24 2022] PUT /identity/v3/projects/6a8b04af7f0842a6863b89832d04610b/users/2ce731482c3d44d29b479593e9e1ff06/roles/a7d20885b6d94928a34b3cc5b3a00006 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:21:24.323126 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1932b13-e56d-4532-bd8f-04bebdeb9805 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:21:24.333699 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1932b13-e56d-4532-bd8f-04bebdeb9805 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:21:24.391245 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72db7c25-c840-4116-bbeb-237d752c5e20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:21:24.391978 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72db7c25-c840-4116-bbeb-237d752c5e20 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:24.392269 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72db7c25-c840-4116-bbeb-237d752c5e20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:24.392559 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72db7c25-c840-4116-bbeb-237d752c5e20 admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:24.393406 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72db7c25-c840-4116-bbeb-237d752c5e20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:21:24.395012 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72db7c25-c840-4116-bbeb-237d752c5e20 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:21:24.409394 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1266/2530] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Jul 25 12:21:24 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:21:24.410495 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddbabea-8cc8-4528-9ae3-c764f83dd250 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:21:24.422403 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddbabea-8cc8-4528-9ae3-c764f83dd250 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:21:24.427055 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853c1f6a-597b-4f79-84f9-594aecd9239b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:21:24.427970 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853c1f6a-597b-4f79-84f9-594aecd9239b admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:24.428258 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853c1f6a-597b-4f79-84f9-594aecd9239b admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:24.428536 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853c1f6a-597b-4f79-84f9-594aecd9239b admin admin] PATH_INFO: `/v3/projects/6a8b04af7f0842a6863b89832d04610b/users/2ce731482c3d44d29b479593e9e1ff06/roles/a7d20885b6d94928a34b3cc5b3a00006` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:24.430547 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853c1f6a-597b-4f79-84f9-594aecd9239b admin admin] RBAC: Authorizing `identity:create_grant(project_id=6a8b04af7f0842a6863b89832d04610b, user_id=2ce731482c3d44d29b479593e9e1ff06, role_id=a7d20885b6d94928a34b3cc5b3a00006)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:21:24.433201 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853c1f6a-597b-4f79-84f9-594aecd9239b admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:21:24.444263 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1265/2531] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Jul 25 12:21:24 2022] PUT /identity/v3/projects/6a8b04af7f0842a6863b89832d04610b/users/2ce731482c3d44d29b479593e9e1ff06/roles/a7d20885b6d94928a34b3cc5b3a00006 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 12:21:24.454413 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9be571b-d57b-49b0-8f9d-cfcaf0a6a8c1 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:24.454708 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9be571b-d57b-49b0-8f9d-cfcaf0a6a8c1 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:24.454992 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9be571b-d57b-49b0-8f9d-cfcaf0a6a8c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:24.483536 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-f9be571b-d57b-49b0-8f9d-cfcaf0a6a8c1 None None] MFA Rules not processed for user `2ce731482c3d44d29b479593e9e1ff06`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:21:24.484777 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-f9be571b-d57b-49b0-8f9d-cfcaf0a6a8c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:21:24.552365 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1267/2532] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:21:24 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:21:24.564394 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853c1f6a-597b-4f79-84f9-594aecd9239b admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:21:24.566157 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-853c1f6a-597b-4f79-84f9-594aecd9239b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:21:24.625099 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853c1f6a-597b-4f79-84f9-594aecd9239b admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:21:24.628717 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d89549-3a2f-4ecd-b878-a6be3cd3c5d4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f89ce6f0cf04373a414d927fa0f8a7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:21:24.629359 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d89549-3a2f-4ecd-b878-a6be3cd3c5d4 service cinder] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:24.629647 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d89549-3a2f-4ecd-b878-a6be3cd3c5d4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:24.629927 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d89549-3a2f-4ecd-b878-a6be3cd3c5d4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:24.641340 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d89549-3a2f-4ecd-b878-a6be3cd3c5d4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:21:24.642790 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d89549-3a2f-4ecd-b878-a6be3cd3c5d4 service cinder] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:21:24.653119 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1266/2533] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:21:24 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:21:24.968924 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9be571b-d57b-49b0-8f9d-cfcaf0a6a8c1 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:21:24.978041 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9be571b-d57b-49b0-8f9d-cfcaf0a6a8c1 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:21:24.983002 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249a9249-5898-43ec-993c-93706b92e981 tempest-VolumeQuotaClassesTest-1088890400 tempest-VolumeQuotaClassesTest-1088890400-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ce731482c3d44d29b479593e9e1ff06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a8b04af7f0842a6863b89832d04610b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:21:24.983698 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249a9249-5898-43ec-993c-93706b92e981 tempest-VolumeQuotaClassesTest-1088890400 tempest-VolumeQuotaClassesTest-1088890400-project-admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:24.983994 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249a9249-5898-43ec-993c-93706b92e981 tempest-VolumeQuotaClassesTest-1088890400 tempest-VolumeQuotaClassesTest-1088890400-project-admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:24.984281 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249a9249-5898-43ec-993c-93706b92e981 tempest-VolumeQuotaClassesTest-1088890400 tempest-VolumeQuotaClassesTest-1088890400-project-admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:24.985045 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249a9249-5898-43ec-993c-93706b92e981 tempest-VolumeQuotaClassesTest-1088890400 tempest-VolumeQuotaClassesTest-1088890400-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:21:24.985365 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249a9249-5898-43ec-993c-93706b92e981 tempest-VolumeQuotaClassesTest-1088890400 tempest-VolumeQuotaClassesTest-1088890400-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:21:24.986978 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249a9249-5898-43ec-993c-93706b92e981 tempest-VolumeQuotaClassesTest-1088890400 tempest-VolumeQuotaClassesTest-1088890400-project-admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:21:24.998554 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1268/2534] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Jul 25 12:21:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:21:25.002736 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d89549-3a2f-4ecd-b878-a6be3cd3c5d4 service cinder] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:21:25.012364 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d89549-3a2f-4ecd-b878-a6be3cd3c5d4 service cinder] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:21:25.016893 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdc5371-ae60-4220-b4fa-d4b7edcaca61 tempest-VolumeQuotaClassesTest-1088890400 tempest-VolumeQuotaClassesTest-1088890400-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ce731482c3d44d29b479593e9e1ff06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a8b04af7f0842a6863b89832d04610b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:21:25.017535 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdc5371-ae60-4220-b4fa-d4b7edcaca61 tempest-VolumeQuotaClassesTest-1088890400 tempest-VolumeQuotaClassesTest-1088890400-project-admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:25.017830 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdc5371-ae60-4220-b4fa-d4b7edcaca61 tempest-VolumeQuotaClassesTest-1088890400 tempest-VolumeQuotaClassesTest-1088890400-project-admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:25.018113 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdc5371-ae60-4220-b4fa-d4b7edcaca61 tempest-VolumeQuotaClassesTest-1088890400 tempest-VolumeQuotaClassesTest-1088890400-project-admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:25.018810 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bdc5371-ae60-4220-b4fa-d4b7edcaca61 tempest-VolumeQuotaClassesTest-1088890400 tempest-VolumeQuotaClassesTest-1088890400-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:21:25.020754 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bdc5371-ae60-4220-b4fa-d4b7edcaca61 tempest-VolumeQuotaClassesTest-1088890400 tempest-VolumeQuotaClassesTest-1088890400-project-admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:21:25.030741 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:21:25.050506 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1267/2535] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Jul 25 12:21:25 2022] POST /identity/v3/projects => generated 354 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 12:21:25.054727 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249a9249-5898-43ec-993c-93706b92e981 tempest-VolumeQuotaClassesTest-1088890400 tempest-VolumeQuotaClassesTest-1088890400-project-admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:21:25.064818 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249a9249-5898-43ec-993c-93706b92e981 tempest-VolumeQuotaClassesTest-1088890400 tempest-VolumeQuotaClassesTest-1088890400-project-admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:21:25.088882 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:21:25.126297 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38870e46-3b42-4b53-87a0-208a57678627 tempest-VolumeQuotaClassesTest-1088890400 tempest-VolumeQuotaClassesTest-1088890400-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ce731482c3d44d29b479593e9e1ff06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a8b04af7f0842a6863b89832d04610b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:21:25.126978 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38870e46-3b42-4b53-87a0-208a57678627 tempest-VolumeQuotaClassesTest-1088890400 tempest-VolumeQuotaClassesTest-1088890400-project-admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:25.127277 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38870e46-3b42-4b53-87a0-208a57678627 tempest-VolumeQuotaClassesTest-1088890400 tempest-VolumeQuotaClassesTest-1088890400-project-admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:25.127587 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38870e46-3b42-4b53-87a0-208a57678627 tempest-VolumeQuotaClassesTest-1088890400 tempest-VolumeQuotaClassesTest-1088890400-project-admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:25.128322 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38870e46-3b42-4b53-87a0-208a57678627 tempest-VolumeQuotaClassesTest-1088890400 tempest-VolumeQuotaClassesTest-1088890400-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:21:25.128643 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38870e46-3b42-4b53-87a0-208a57678627 tempest-VolumeQuotaClassesTest-1088890400 tempest-VolumeQuotaClassesTest-1088890400-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:21:25.130282 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38870e46-3b42-4b53-87a0-208a57678627 tempest-VolumeQuotaClassesTest-1088890400 tempest-VolumeQuotaClassesTest-1088890400-project-admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:21:25.142259 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1269/2536] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Jul 25 12:21:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:21:25.184747 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdc5371-ae60-4220-b4fa-d4b7edcaca61 tempest-VolumeQuotaClassesTest-1088890400 tempest-VolumeQuotaClassesTest-1088890400-project-admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:21:25.196983 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdc5371-ae60-4220-b4fa-d4b7edcaca61 tempest-VolumeQuotaClassesTest-1088890400 tempest-VolumeQuotaClassesTest-1088890400-project-admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:21:25.201627 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06888d67-758d-4cb9-8b10-97f3e678d3df tempest-VolumeQuotaClassesTest-1088890400 tempest-VolumeQuotaClassesTest-1088890400-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ce731482c3d44d29b479593e9e1ff06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a8b04af7f0842a6863b89832d04610b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:21:25.202484 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06888d67-758d-4cb9-8b10-97f3e678d3df tempest-VolumeQuotaClassesTest-1088890400 tempest-VolumeQuotaClassesTest-1088890400-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:25.202777 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06888d67-758d-4cb9-8b10-97f3e678d3df tempest-VolumeQuotaClassesTest-1088890400 tempest-VolumeQuotaClassesTest-1088890400-project-admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:25.203064 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06888d67-758d-4cb9-8b10-97f3e678d3df tempest-VolumeQuotaClassesTest-1088890400 tempest-VolumeQuotaClassesTest-1088890400-project-admin] PATH_INFO: `/v3/projects/7c83ac1a1c7046e7ba00b7da4ed2262b` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:25.204258 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06888d67-758d-4cb9-8b10-97f3e678d3df tempest-VolumeQuotaClassesTest-1088890400 tempest-VolumeQuotaClassesTest-1088890400-project-admin] RBAC: Authorizing `identity:delete_project(project_id=7c83ac1a1c7046e7ba00b7da4ed2262b)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:21:25.206166 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06888d67-758d-4cb9-8b10-97f3e678d3df tempest-VolumeQuotaClassesTest-1088890400 tempest-VolumeQuotaClassesTest-1088890400-project-admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:21:25.251888 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-06888d67-758d-4cb9-8b10-97f3e678d3df tempest-VolumeQuotaClassesTest-1088890400 tempest-VolumeQuotaClassesTest-1088890400-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7c83ac1a1c7046e7ba00b7da4ed2262b', 'request_id': 'req-06888d67-758d-4cb9-8b10-97f3e678d3df'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:25.259900 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-06888d67-758d-4cb9-8b10-97f3e678d3df tempest-VolumeQuotaClassesTest-1088890400 tempest-VolumeQuotaClassesTest-1088890400-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7c83ac1a1c7046e7ba00b7da4ed2262b', 'request_id': 'req-06888d67-758d-4cb9-8b10-97f3e678d3df'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:25.265569 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-06888d67-758d-4cb9-8b10-97f3e678d3df tempest-VolumeQuotaClassesTest-1088890400 tempest-VolumeQuotaClassesTest-1088890400-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7c83ac1a1c7046e7ba00b7da4ed2262b', 'request_id': 'req-06888d67-758d-4cb9-8b10-97f3e678d3df'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:25.276834 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-06888d67-758d-4cb9-8b10-97f3e678d3df tempest-VolumeQuotaClassesTest-1088890400 tempest-VolumeQuotaClassesTest-1088890400-project-admin] The token cache is being invalidate because project 7c83ac1a1c7046e7ba00b7da4ed2262b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 12:21:25.277130 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-06888d67-758d-4cb9-8b10-97f3e678d3df tempest-VolumeQuotaClassesTest-1088890400 tempest-VolumeQuotaClassesTest-1088890400-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:25.277836 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-06888d67-758d-4cb9-8b10-97f3e678d3df tempest-VolumeQuotaClassesTest-1088890400 tempest-VolumeQuotaClassesTest-1088890400-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:25.279368 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1268/2537] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Jul 25 12:21:25 2022] DELETE /identity/v3/projects/7c83ac1a1c7046e7ba00b7da4ed2262b => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:21:25.472795 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38870e46-3b42-4b53-87a0-208a57678627 tempest-VolumeQuotaClassesTest-1088890400 tempest-VolumeQuotaClassesTest-1088890400-project-admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:21:25.474835 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-38870e46-3b42-4b53-87a0-208a57678627 tempest-VolumeQuotaClassesTest-1088890400 tempest-VolumeQuotaClassesTest-1088890400-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:21:25.541859 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38870e46-3b42-4b53-87a0-208a57678627 tempest-VolumeQuotaClassesTest-1088890400 tempest-VolumeQuotaClassesTest-1088890400-project-admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:21:25.600080 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977f7937-0faa-4a36-834e-9026355ec3d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:21:25.601449 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977f7937-0faa-4a36-834e-9026355ec3d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:25.601739 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977f7937-0faa-4a36-834e-9026355ec3d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:25.602023 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977f7937-0faa-4a36-834e-9026355ec3d6 admin admin] PATH_INFO: `/v3/users/98b073de38f14638a767b5b3400bb40c` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:25.603449 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977f7937-0faa-4a36-834e-9026355ec3d6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=98b073de38f14638a767b5b3400bb40c)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:21:25.605488 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977f7937-0faa-4a36-834e-9026355ec3d6 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:21:25.656809 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-977f7937-0faa-4a36-834e-9026355ec3d6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '98b073de38f14638a767b5b3400bb40c'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:25.657468 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-977f7937-0faa-4a36-834e-9026355ec3d6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '98b073de38f14638a767b5b3400bb40c'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:25.665408 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-977f7937-0faa-4a36-834e-9026355ec3d6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '98b073de38f14638a767b5b3400bb40c'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:25.674088 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:21:25.679830 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-977f7937-0faa-4a36-834e-9026355ec3d6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '98b073de38f14638a767b5b3400bb40c'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:25.687898 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-977f7937-0faa-4a36-834e-9026355ec3d6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '98b073de38f14638a767b5b3400bb40c'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:25.693925 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1270/2538] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Jul 25 12:21:25 2022] DELETE /identity/v3/users/98b073de38f14638a767b5b3400bb40c => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:21:25.698950 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5964fdf-2cbd-4b02-96aa-048b72aa3085 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:25.699430 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5964fdf-2cbd-4b02-96aa-048b72aa3085 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:25.699575 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5964fdf-2cbd-4b02-96aa-048b72aa3085 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:25.718635 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-d5964fdf-2cbd-4b02-96aa-048b72aa3085 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:21:25.719868 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-d5964fdf-2cbd-4b02-96aa-048b72aa3085 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:21:25.777683 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1269/2539] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:21:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:21:25.787114 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977f7937-0faa-4a36-834e-9026355ec3d6 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:21:25.788848 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-977f7937-0faa-4a36-834e-9026355ec3d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:21:25.846197 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977f7937-0faa-4a36-834e-9026355ec3d6 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:21:25.849972 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b295cba-ed77-4a09-a835-b1e4e46d88b5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51733781201243e8abf9eca39b207750', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:21:25.850615 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b295cba-ed77-4a09-a835-b1e4e46d88b5 service neutron] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:25.850903 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b295cba-ed77-4a09-a835-b1e4e46d88b5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:25.851186 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b295cba-ed77-4a09-a835-b1e4e46d88b5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:25.860672 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b295cba-ed77-4a09-a835-b1e4e46d88b5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:21:25.862094 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b295cba-ed77-4a09-a835-b1e4e46d88b5 service neutron] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:21:25.872069 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1271/2540] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:21:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:21:26.136279 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5964fdf-2cbd-4b02-96aa-048b72aa3085 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:21:26.137915 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-d5964fdf-2cbd-4b02-96aa-048b72aa3085 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:21:26.155573 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5964fdf-2cbd-4b02-96aa-048b72aa3085 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:21:26.160195 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd4e061-5174-48a8-92bd-48d1f075cb13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:21:26.161084 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd4e061-5174-48a8-92bd-48d1f075cb13 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:26.161372 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd4e061-5174-48a8-92bd-48d1f075cb13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:26.161710 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd4e061-5174-48a8-92bd-48d1f075cb13 admin admin] PATH_INFO: `/v3/projects/82b2b9991d98449897da4a889733c9c7` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:26.162835 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd4e061-5174-48a8-92bd-48d1f075cb13 admin admin] RBAC: Authorizing `identity:delete_project(project_id=82b2b9991d98449897da4a889733c9c7)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:21:26.164854 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd4e061-5174-48a8-92bd-48d1f075cb13 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:21:26.212177 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-ddd4e061-5174-48a8-92bd-48d1f075cb13 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '82b2b9991d98449897da4a889733c9c7', 'request_id': 'req-ddd4e061-5174-48a8-92bd-48d1f075cb13'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:26.219380 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-ddd4e061-5174-48a8-92bd-48d1f075cb13 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '82b2b9991d98449897da4a889733c9c7', 'request_id': 'req-ddd4e061-5174-48a8-92bd-48d1f075cb13'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:26.224929 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-ddd4e061-5174-48a8-92bd-48d1f075cb13 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '82b2b9991d98449897da4a889733c9c7', 'request_id': 'req-ddd4e061-5174-48a8-92bd-48d1f075cb13'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:26.226933 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=83,keystone:INSERT=3,keystone:DELETE=14 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:21:26.238221 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-ddd4e061-5174-48a8-92bd-48d1f075cb13 admin admin] The token cache is being invalidate because project 82b2b9991d98449897da4a889733c9c7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 12:21:26.238529 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-ddd4e061-5174-48a8-92bd-48d1f075cb13 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:26.239453 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-ddd4e061-5174-48a8-92bd-48d1f075cb13 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:26.241090 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1270/2541] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Jul 25 12:21:26 2022] DELETE /identity/v3/projects/82b2b9991d98449897da4a889733c9c7 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 12:21:26.245573 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b295cba-ed77-4a09-a835-b1e4e46d88b5 service neutron] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:21:26.255199 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b295cba-ed77-4a09-a835-b1e4e46d88b5 service neutron] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:21:26.313692 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3889b86-d6ff-421c-9269-240721445fa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:21:26.314563 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3889b86-d6ff-421c-9269-240721445fa1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:26.314848 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3889b86-d6ff-421c-9269-240721445fa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:26.315128 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3889b86-d6ff-421c-9269-240721445fa1 admin admin] PATH_INFO: `/v3/users/2ce731482c3d44d29b479593e9e1ff06` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:26.316364 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3889b86-d6ff-421c-9269-240721445fa1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2ce731482c3d44d29b479593e9e1ff06)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:21:26.318282 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3889b86-d6ff-421c-9269-240721445fa1 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:21:26.364017 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-f3889b86-d6ff-421c-9269-240721445fa1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2ce731482c3d44d29b479593e9e1ff06'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:26.364670 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-f3889b86-d6ff-421c-9269-240721445fa1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2ce731482c3d44d29b479593e9e1ff06'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:26.372816 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-f3889b86-d6ff-421c-9269-240721445fa1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2ce731482c3d44d29b479593e9e1ff06'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:26.385866 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-f3889b86-d6ff-421c-9269-240721445fa1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2ce731482c3d44d29b479593e9e1ff06'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:26.392905 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-f3889b86-d6ff-421c-9269-240721445fa1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2ce731482c3d44d29b479593e9e1ff06'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:26.398872 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1272/2542] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Jul 25 12:21:26 2022] DELETE /identity/v3/users/2ce731482c3d44d29b479593e9e1ff06 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:21:26.678870 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd4e061-5174-48a8-92bd-48d1f075cb13 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:21:26.680839 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-ddd4e061-5174-48a8-92bd-48d1f075cb13 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:21:26.700651 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd4e061-5174-48a8-92bd-48d1f075cb13 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:21:26.766266 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dd044b-0f07-4ab0-9554-4a2a3952aadc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:21:26.767211 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dd044b-0f07-4ab0-9554-4a2a3952aadc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:26.767592 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dd044b-0f07-4ab0-9554-4a2a3952aadc admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:26.768087 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dd044b-0f07-4ab0-9554-4a2a3952aadc admin admin] PATH_INFO: `/v3/projects/6a8b04af7f0842a6863b89832d04610b` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:26.777087 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0dd044b-0f07-4ab0-9554-4a2a3952aadc admin admin] RBAC: Authorizing `identity:delete_project(project_id=6a8b04af7f0842a6863b89832d04610b)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:21:26.779257 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0dd044b-0f07-4ab0-9554-4a2a3952aadc admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:21:26.839838 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-c0dd044b-0f07-4ab0-9554-4a2a3952aadc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6a8b04af7f0842a6863b89832d04610b', 'request_id': 'req-c0dd044b-0f07-4ab0-9554-4a2a3952aadc'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:26.847439 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-c0dd044b-0f07-4ab0-9554-4a2a3952aadc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6a8b04af7f0842a6863b89832d04610b', 'request_id': 'req-c0dd044b-0f07-4ab0-9554-4a2a3952aadc'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:26.854171 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-c0dd044b-0f07-4ab0-9554-4a2a3952aadc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6a8b04af7f0842a6863b89832d04610b', 'request_id': 'req-c0dd044b-0f07-4ab0-9554-4a2a3952aadc'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:26.859723 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3889b86-d6ff-421c-9269-240721445fa1 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:21:26.861190 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-f3889b86-d6ff-421c-9269-240721445fa1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:21:26.865838 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-c0dd044b-0f07-4ab0-9554-4a2a3952aadc admin admin] The token cache is being invalidate because project 6a8b04af7f0842a6863b89832d04610b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 12:21:26.866159 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-c0dd044b-0f07-4ab0-9554-4a2a3952aadc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:26.867331 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-c0dd044b-0f07-4ab0-9554-4a2a3952aadc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:26.869107 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1271/2543] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Jul 25 12:21:26 2022] DELETE /identity/v3/projects/6a8b04af7f0842a6863b89832d04610b => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:21:26.873010 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:21:26.874381 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fd1807-2e5d-4536-8298-6d2936bd23ab None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:26.874674 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fd1807-2e5d-4536-8298-6d2936bd23ab None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:26.874960 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fd1807-2e5d-4536-8298-6d2936bd23ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:26.876775 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3889b86-d6ff-421c-9269-240721445fa1 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:21:26.896289 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-c2fd1807-2e5d-4536-8298-6d2936bd23ab None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:21:26.897491 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-c2fd1807-2e5d-4536-8298-6d2936bd23ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:21:26.937033 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce1316a-a013-4786-9a1f-b3b07f7651b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:21:26.937964 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce1316a-a013-4786-9a1f-b3b07f7651b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:26.938258 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce1316a-a013-4786-9a1f-b3b07f7651b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:26.938544 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce1316a-a013-4786-9a1f-b3b07f7651b4 admin admin] PATH_INFO: `/v3/users/b507f83f1f6d406ba9a686486a8353a6` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:26.939768 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce1316a-a013-4786-9a1f-b3b07f7651b4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b507f83f1f6d406ba9a686486a8353a6)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:21:26.941700 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce1316a-a013-4786-9a1f-b3b07f7651b4 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:21:26.958757 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1272/2544] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:21:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 12:21:26.967772 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191bf0d5-fcb1-4062-a550-f8e6a0dfce8e None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:26.968099 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191bf0d5-fcb1-4062-a550-f8e6a0dfce8e None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:26.968516 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191bf0d5-fcb1-4062-a550-f8e6a0dfce8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:26.990082 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-191bf0d5-fcb1-4062-a550-f8e6a0dfce8e None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:21:26.990977 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-bce1316a-a013-4786-9a1f-b3b07f7651b4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b507f83f1f6d406ba9a686486a8353a6'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:26.991303 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-191bf0d5-fcb1-4062-a550-f8e6a0dfce8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:21:26.992169 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-bce1316a-a013-4786-9a1f-b3b07f7651b4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b507f83f1f6d406ba9a686486a8353a6'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:27.000014 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-bce1316a-a013-4786-9a1f-b3b07f7651b4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b507f83f1f6d406ba9a686486a8353a6'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:27.001567 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1273/2545] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:21:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 12:21:27.005769 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191bf0d5-fcb1-4062-a550-f8e6a0dfce8e None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:21:27.013198 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-bce1316a-a013-4786-9a1f-b3b07f7651b4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b507f83f1f6d406ba9a686486a8353a6'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:27.015396 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191bf0d5-fcb1-4062-a550-f8e6a0dfce8e None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:21:27.019866 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-bce1316a-a013-4786-9a1f-b3b07f7651b4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b507f83f1f6d406ba9a686486a8353a6'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:27.020029 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8104b8a-efe2-4cd5-b331-553dc82b8b76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:21:27.020662 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8104b8a-efe2-4cd5-b331-553dc82b8b76 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:27.020949 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8104b8a-efe2-4cd5-b331-553dc82b8b76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:27.021235 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8104b8a-efe2-4cd5-b331-553dc82b8b76 admin admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:27.021955 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8104b8a-efe2-4cd5-b331-553dc82b8b76 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:21:27.022265 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8104b8a-efe2-4cd5-b331-553dc82b8b76 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:21:27.023916 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8104b8a-efe2-4cd5-b331-553dc82b8b76 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:21:27.025986 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1273/2546] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Jul 25 12:21:26 2022] DELETE /identity/v3/users/b507f83f1f6d406ba9a686486a8353a6 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:21:27.030421 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c39854-6592-479d-8c37-3ed0aae3ec72 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:27.030706 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c39854-6592-479d-8c37-3ed0aae3ec72 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:27.031023 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c39854-6592-479d-8c37-3ed0aae3ec72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:27.036358 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1274/2547] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Jul 25 12:21:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 12:21:27.040941 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8104b8a-efe2-4cd5-b331-553dc82b8b76 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:21:27.042599 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-d8104b8a-efe2-4cd5-b331-553dc82b8b76 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:21:27.056790 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8104b8a-efe2-4cd5-b331-553dc82b8b76 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:21:27.060826 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:21:27.127868 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ed381b-2f14-4f8a-97ba-cf246104adbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:21:27.128565 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ed381b-2f14-4f8a-97ba-cf246104adbf admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:27.128856 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ed381b-2f14-4f8a-97ba-cf246104adbf admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:27.129141 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ed381b-2f14-4f8a-97ba-cf246104adbf admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:27.129858 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ed381b-2f14-4f8a-97ba-cf246104adbf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:21:27.131844 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ed381b-2f14-4f8a-97ba-cf246104adbf admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:21:27.168248 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1275/2548] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Jul 25 12:21:27 2022] POST /identity/v3/projects => generated 407 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 12:21:27.170033 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ed381b-2f14-4f8a-97ba-cf246104adbf admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:21:27.180767 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-a6c39854-6592-479d-8c37-3ed0aae3ec72 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:21:27.181999 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-a6c39854-6592-479d-8c37-3ed0aae3ec72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:21:27.250065 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ed381b-2f14-4f8a-97ba-cf246104adbf admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:21:27.254778 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3db11d-9aba-4d57-9c76-594943a42ead admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:21:27.255532 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3db11d-9aba-4d57-9c76-594943a42ead admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:27.255923 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3db11d-9aba-4d57-9c76-594943a42ead admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:27.256275 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3db11d-9aba-4d57-9c76-594943a42ead admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:27.257710 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3db11d-9aba-4d57-9c76-594943a42ead admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:21:27.259804 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3db11d-9aba-4d57-9c76-594943a42ead admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:21:27.263527 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1274/2549] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:21:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 12:21:27.276369 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c39854-6592-479d-8c37-3ed0aae3ec72 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:21:27.277830 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-a6c39854-6592-479d-8c37-3ed0aae3ec72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:21:27.289517 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1276/2550] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Jul 25 12:21:27 2022] POST /identity/v3/users => generated 304 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 12:21:27.290797 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3db11d-9aba-4d57-9c76-594943a42ead admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:21:27.301427 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3db11d-9aba-4d57-9c76-594943a42ead admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:21:27.307344 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815240a3-2e42-4d34-b788-3634a9955ccc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:21:27.308267 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815240a3-2e42-4d34-b788-3634a9955ccc admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:27.308564 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815240a3-2e42-4d34-b788-3634a9955ccc admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:27.308846 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815240a3-2e42-4d34-b788-3634a9955ccc admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:27.309834 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815240a3-2e42-4d34-b788-3634a9955ccc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:21:27.311830 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815240a3-2e42-4d34-b788-3634a9955ccc admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:21:27.315166 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:21:27.327117 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1277/2551] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Jul 25 12:21:27 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:21:27.328717 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815240a3-2e42-4d34-b788-3634a9955ccc admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:21:27.341055 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815240a3-2e42-4d34-b788-3634a9955ccc admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:21:27.345872 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2df1e7-2914-40d1-b742-2ab4cf5b0c52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:21:27.346538 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2df1e7-2914-40d1-b742-2ab4cf5b0c52 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:27.346824 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2df1e7-2914-40d1-b742-2ab4cf5b0c52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:27.347102 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2df1e7-2914-40d1-b742-2ab4cf5b0c52 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:27.347871 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2df1e7-2914-40d1-b742-2ab4cf5b0c52 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:21:27.349812 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2df1e7-2914-40d1-b742-2ab4cf5b0c52 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:21:27.351911 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c39854-6592-479d-8c37-3ed0aae3ec72 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:21:27.357738 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1c4455-50fa-4daf-96ac-ade9030a3126 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51733781201243e8abf9eca39b207750', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:21:27.358802 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1c4455-50fa-4daf-96ac-ade9030a3126 service neutron] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:27.359208 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1c4455-50fa-4daf-96ac-ade9030a3126 service neutron] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:27.359482 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1c4455-50fa-4daf-96ac-ade9030a3126 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:27.363034 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1278/2552] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Jul 25 12:21:27 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 12:21:27.367879 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2df1e7-2914-40d1-b742-2ab4cf5b0c52 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:21:27.371590 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1c4455-50fa-4daf-96ac-ade9030a3126 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:21:27.373159 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1c4455-50fa-4daf-96ac-ade9030a3126 service neutron] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:21:27.379955 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2df1e7-2914-40d1-b742-2ab4cf5b0c52 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:21:27.383188 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1275/2553] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:21:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:21:27.385628 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3dfba71-2059-4f9b-9b8e-9133b6da0603 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:21:27.386544 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dfba71-2059-4f9b-9b8e-9133b6da0603 admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:27.386824 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dfba71-2059-4f9b-9b8e-9133b6da0603 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:27.387102 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dfba71-2059-4f9b-9b8e-9133b6da0603 admin admin] PATH_INFO: `/v3/projects/7b3f8796504c47fab663669f741348dc/users/89aa9d11a1d644a180916cefbbe8dbf6/roles/472313ae69174578824df0bed6390744` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:27.407615 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3dfba71-2059-4f9b-9b8e-9133b6da0603 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b3f8796504c47fab663669f741348dc, user_id=89aa9d11a1d644a180916cefbbe8dbf6, role_id=472313ae69174578824df0bed6390744)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:21:27.411960 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3dfba71-2059-4f9b-9b8e-9133b6da0603 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:21:27.437233 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1279/2554] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Jul 25 12:21:27 2022] PUT /identity/v3/projects/7b3f8796504c47fab663669f741348dc/users/89aa9d11a1d644a180916cefbbe8dbf6/roles/472313ae69174578824df0bed6390744 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 12:21:27.449943 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a669516-e554-4a49-bdaa-09685638e3f9 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:27.450390 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a669516-e554-4a49-bdaa-09685638e3f9 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:27.451029 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a669516-e554-4a49-bdaa-09685638e3f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:27.484273 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-6a669516-e554-4a49-bdaa-09685638e3f9 None None] MFA Rules not processed for user `89aa9d11a1d644a180916cefbbe8dbf6`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:21:27.485561 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-6a669516-e554-4a49-bdaa-09685638e3f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:21:27.567869 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3dfba71-2059-4f9b-9b8e-9133b6da0603 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:21:27.571224 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1276/2555] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:21:27 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:21:27.657778 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:21:27.684659 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3dfba71-2059-4f9b-9b8e-9133b6da0603 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:21:27.689765 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe9ada2-f16c-4bc1-8091-f613809b1731 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:21:27.690529 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe9ada2-f16c-4bc1-8091-f613809b1731 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:27.690889 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe9ada2-f16c-4bc1-8091-f613809b1731 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:27.691246 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe9ada2-f16c-4bc1-8091-f613809b1731 admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:27.692131 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe9ada2-f16c-4bc1-8091-f613809b1731 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:21:27.694170 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe9ada2-f16c-4bc1-8091-f613809b1731 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:21:27.724546 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1280/2556] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Jul 25 12:21:27 2022] POST /identity/v3/projects => generated 407 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:21:27.729142 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a669516-e554-4a49-bdaa-09685638e3f9 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:21:27.743141 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a669516-e554-4a49-bdaa-09685638e3f9 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:21:27.756688 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe9ada2-f16c-4bc1-8091-f613809b1731 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:21:27.759036 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-7fe9ada2-f16c-4bc1-8091-f613809b1731 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:21:27.831389 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8ad1a8-4681-41f8-a8b9-3fb02178f404 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:21:27.832101 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8ad1a8-4681-41f8-a8b9-3fb02178f404 admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:27.832390 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8ad1a8-4681-41f8-a8b9-3fb02178f404 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:27.832672 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8ad1a8-4681-41f8-a8b9-3fb02178f404 admin admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:27.833366 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8ad1a8-4681-41f8-a8b9-3fb02178f404 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:21:27.835406 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8ad1a8-4681-41f8-a8b9-3fb02178f404 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:21:27.857313 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe9ada2-f16c-4bc1-8091-f613809b1731 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:21:27.860407 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1277/2557] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Jul 25 12:21:27 2022] POST /identity/v3/users => generated 310 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:21:27.863696 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d89682-b307-4f3f-a22b-1d85b21beac8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:21:27.863696 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d89682-b307-4f3f-a22b-1d85b21beac8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:27.863696 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d89682-b307-4f3f-a22b-1d85b21beac8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:27.863696 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d89682-b307-4f3f-a22b-1d85b21beac8 admin admin] PATH_INFO: `/v3/projects/39471a6491054b2fb2e3f126159fdede` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:27.864663 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d89682-b307-4f3f-a22b-1d85b21beac8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=39471a6491054b2fb2e3f126159fdede)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:21:27.864778 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8ad1a8-4681-41f8-a8b9-3fb02178f404 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:21:27.867228 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d89682-b307-4f3f-a22b-1d85b21beac8 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:21:27.875449 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8ad1a8-4681-41f8-a8b9-3fb02178f404 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:21:27.881084 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688c3edc-97f6-45c7-b13f-9c58dbc00e40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:21:27.881740 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688c3edc-97f6-45c7-b13f-9c58dbc00e40 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:27.882069 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688c3edc-97f6-45c7-b13f-9c58dbc00e40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:27.882349 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688c3edc-97f6-45c7-b13f-9c58dbc00e40 admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:27.883121 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688c3edc-97f6-45c7-b13f-9c58dbc00e40 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:21:27.884754 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688c3edc-97f6-45c7-b13f-9c58dbc00e40 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:21:27.895886 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1278/2558] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Jul 25 12:21:27 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:21:27.901166 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688c3edc-97f6-45c7-b13f-9c58dbc00e40 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:21:27.911587 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688c3edc-97f6-45c7-b13f-9c58dbc00e40 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:21:27.918611 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-82d89682-b307-4f3f-a22b-1d85b21beac8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '39471a6491054b2fb2e3f126159fdede', 'request_id': 'req-82d89682-b307-4f3f-a22b-1d85b21beac8'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:27.926384 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-82d89682-b307-4f3f-a22b-1d85b21beac8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '39471a6491054b2fb2e3f126159fdede', 'request_id': 'req-82d89682-b307-4f3f-a22b-1d85b21beac8'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:27.934379 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-82d89682-b307-4f3f-a22b-1d85b21beac8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '39471a6491054b2fb2e3f126159fdede', 'request_id': 'req-82d89682-b307-4f3f-a22b-1d85b21beac8'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:27.947105 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-82d89682-b307-4f3f-a22b-1d85b21beac8 admin admin] The token cache is being invalidate because project 39471a6491054b2fb2e3f126159fdede was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 12:21:27.947500 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-82d89682-b307-4f3f-a22b-1d85b21beac8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:27.948093 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-82d89682-b307-4f3f-a22b-1d85b21beac8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:27.948972 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:21:27.949669 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1281/2559] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Jul 25 12:21:27 2022] DELETE /identity/v3/projects/39471a6491054b2fb2e3f126159fdede => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 12:21:27.953711 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d89682-b307-4f3f-a22b-1d85b21beac8 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:21:27.955278 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-82d89682-b307-4f3f-a22b-1d85b21beac8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:21:27.977685 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078d7756-a51d-4747-81c8-06d332fad7db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:21:27.978571 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078d7756-a51d-4747-81c8-06d332fad7db admin admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:27.978857 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078d7756-a51d-4747-81c8-06d332fad7db admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:27.979132 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078d7756-a51d-4747-81c8-06d332fad7db admin admin] PATH_INFO: `/v3/projects/140ca86474e84c05a2b683060f71837c/users/dc447433b7084f5c9b259a02637aab51/roles/a7d20885b6d94928a34b3cc5b3a00006` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:27.993545 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078d7756-a51d-4747-81c8-06d332fad7db admin admin] RBAC: Authorizing `identity:create_grant(project_id=140ca86474e84c05a2b683060f71837c, user_id=dc447433b7084f5c9b259a02637aab51, role_id=a7d20885b6d94928a34b3cc5b3a00006)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:21:27.996212 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078d7756-a51d-4747-81c8-06d332fad7db admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:21:28.009294 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1279/2560] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Jul 25 12:21:27 2022] PUT /identity/v3/projects/140ca86474e84c05a2b683060f71837c/users/dc447433b7084f5c9b259a02637aab51/roles/a7d20885b6d94928a34b3cc5b3a00006 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:21:28.010165 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:21:28.013317 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078d7756-a51d-4747-81c8-06d332fad7db admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:21:28.015037 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-078d7756-a51d-4747-81c8-06d332fad7db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:21:28.024745 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d89682-b307-4f3f-a22b-1d85b21beac8 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:21:28.078057 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078d7756-a51d-4747-81c8-06d332fad7db admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:21:28.081743 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f259f264-3afd-48b6-91b2-56b92cbcc2ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:21:28.082758 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f259f264-3afd-48b6-91b2-56b92cbcc2ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:28.082925 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e57ac7-c176-4a1d-90e7-d7ef3dce3fca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:21:28.083121 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f259f264-3afd-48b6-91b2-56b92cbcc2ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:28.083517 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f259f264-3afd-48b6-91b2-56b92cbcc2ad admin admin] PATH_INFO: `/v3/users/80b393d9534b4cb182c2b9ad5bed4f83` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:28.083642 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e57ac7-c176-4a1d-90e7-d7ef3dce3fca admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:28.083884 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e57ac7-c176-4a1d-90e7-d7ef3dce3fca admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:28.084168 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e57ac7-c176-4a1d-90e7-d7ef3dce3fca admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:28.084849 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f259f264-3afd-48b6-91b2-56b92cbcc2ad admin admin] RBAC: Authorizing `identity:delete_user(user_id=80b393d9534b4cb182c2b9ad5bed4f83)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:21:28.084954 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e57ac7-c176-4a1d-90e7-d7ef3dce3fca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:21:28.087058 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e57ac7-c176-4a1d-90e7-d7ef3dce3fca admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:21:28.087481 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f259f264-3afd-48b6-91b2-56b92cbcc2ad admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:21:28.096882 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1280/2561] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Jul 25 12:21:28 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:21:28.100917 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e57ac7-c176-4a1d-90e7-d7ef3dce3fca admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:21:28.110455 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e57ac7-c176-4a1d-90e7-d7ef3dce3fca admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:21:28.115510 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a958b77c-e86d-46ae-bc95-8ff290b56fa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:21:28.116962 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a958b77c-e86d-46ae-bc95-8ff290b56fa9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:28.117248 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a958b77c-e86d-46ae-bc95-8ff290b56fa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:28.117521 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a958b77c-e86d-46ae-bc95-8ff290b56fa9 admin admin] PATH_INFO: `/v3/projects/140ca86474e84c05a2b683060f71837c/users/dc447433b7084f5c9b259a02637aab51/roles/a7d20885b6d94928a34b3cc5b3a00006` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:28.119603 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a958b77c-e86d-46ae-bc95-8ff290b56fa9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=140ca86474e84c05a2b683060f71837c, user_id=dc447433b7084f5c9b259a02637aab51, role_id=a7d20885b6d94928a34b3cc5b3a00006)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:21:28.122552 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a958b77c-e86d-46ae-bc95-8ff290b56fa9 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:21:28.133648 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1281/2562] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Jul 25 12:21:28 2022] PUT /identity/v3/projects/140ca86474e84c05a2b683060f71837c/users/dc447433b7084f5c9b259a02637aab51/roles/a7d20885b6d94928a34b3cc5b3a00006 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 12:21:28.137390 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-f259f264-3afd-48b6-91b2-56b92cbcc2ad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '80b393d9534b4cb182c2b9ad5bed4f83'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:28.143436 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22a8827-eb6f-4569-ba1e-bc39e9dcc74e None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:28.143759 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22a8827-eb6f-4569-ba1e-bc39e9dcc74e None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:28.144046 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22a8827-eb6f-4569-ba1e-bc39e9dcc74e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:28.144516 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-f259f264-3afd-48b6-91b2-56b92cbcc2ad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '80b393d9534b4cb182c2b9ad5bed4f83'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:28.152265 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-f259f264-3afd-48b6-91b2-56b92cbcc2ad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '80b393d9534b4cb182c2b9ad5bed4f83'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:28.164380 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-f259f264-3afd-48b6-91b2-56b92cbcc2ad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '80b393d9534b4cb182c2b9ad5bed4f83'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:28.164989 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-f259f264-3afd-48b6-91b2-56b92cbcc2ad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '80b393d9534b4cb182c2b9ad5bed4f83'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:28.171410 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1282/2563] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Jul 25 12:21:27 2022] DELETE /identity/v3/users/80b393d9534b4cb182c2b9ad5bed4f83 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 12:21:28.172464 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-b22a8827-eb6f-4569-ba1e-bc39e9dcc74e None None] MFA Rules not processed for user `dc447433b7084f5c9b259a02637aab51`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:21:28.173801 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-b22a8827-eb6f-4569-ba1e-bc39e9dcc74e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:21:28.245885 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1282/2564] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:21:28 2022] POST /identity/v3/auth/tokens => generated 3341 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 12:21:28.257835 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f259f264-3afd-48b6-91b2-56b92cbcc2ad admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:21:28.259290 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-f259f264-3afd-48b6-91b2-56b92cbcc2ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:21:28.318608 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f259f264-3afd-48b6-91b2-56b92cbcc2ad admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:21:28.323143 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d510572-fb46-4b29-8902-bf8e1987f41e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f89ce6f0cf04373a414d927fa0f8a7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:21:28.323910 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d510572-fb46-4b29-8902-bf8e1987f41e service cinder] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:28.324265 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d510572-fb46-4b29-8902-bf8e1987f41e service cinder] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:28.324616 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d510572-fb46-4b29-8902-bf8e1987f41e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:28.334964 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d510572-fb46-4b29-8902-bf8e1987f41e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:21:28.336468 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d510572-fb46-4b29-8902-bf8e1987f41e service cinder] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:21:28.346155 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1283/2565] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:21:28 2022] GET /identity/v3/auth/tokens => generated 3341 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:21:28.452327 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22a8827-eb6f-4569-ba1e-bc39e9dcc74e None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:21:28.454020 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-b22a8827-eb6f-4569-ba1e-bc39e9dcc74e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:21:28.506072 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d510572-fb46-4b29-8902-bf8e1987f41e service cinder] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:21:28.513480 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:21:28.516777 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d510572-fb46-4b29-8902-bf8e1987f41e service cinder] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:21:28.520790 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e26ece-441d-499c-944a-b75b4ef35134 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f89ce6f0cf04373a414d927fa0f8a7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:21:28.521447 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e26ece-441d-499c-944a-b75b4ef35134 service cinder] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:28.521739 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e26ece-441d-499c-944a-b75b4ef35134 service cinder] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:28.522022 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e26ece-441d-499c-944a-b75b4ef35134 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:28.524110 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-e6e26ece-441d-499c-944a-b75b4ef35134 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:21:28.543177 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:21:28.551616 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22a8827-eb6f-4569-ba1e-bc39e9dcc74e None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:21:28.556935 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bae74f3-c848-43bf-babd-1cac8088921f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:21:28.557872 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bae74f3-c848-43bf-babd-1cac8088921f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:28.558178 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bae74f3-c848-43bf-babd-1cac8088921f admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:28.558442 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bae74f3-c848-43bf-babd-1cac8088921f admin admin] PATH_INFO: `/v3/projects/9cc67c4674b64223ae2a8c9e7de9b6d4` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:28.559689 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bae74f3-c848-43bf-babd-1cac8088921f admin admin] RBAC: Authorizing `identity:delete_project(project_id=9cc67c4674b64223ae2a8c9e7de9b6d4)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:21:28.561595 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bae74f3-c848-43bf-babd-1cac8088921f admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:21:28.588060 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e26ece-441d-499c-944a-b75b4ef35134 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:21:28.589433 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e26ece-441d-499c-944a-b75b4ef35134 service cinder] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:21:28.598811 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1284/2566] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:21:28 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 12:21:28.610240 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-5bae74f3-c848-43bf-babd-1cac8088921f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9cc67c4674b64223ae2a8c9e7de9b6d4', 'request_id': 'req-5bae74f3-c848-43bf-babd-1cac8088921f'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:28.619236 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-5bae74f3-c848-43bf-babd-1cac8088921f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9cc67c4674b64223ae2a8c9e7de9b6d4', 'request_id': 'req-5bae74f3-c848-43bf-babd-1cac8088921f'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:28.625554 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-5bae74f3-c848-43bf-babd-1cac8088921f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9cc67c4674b64223ae2a8c9e7de9b6d4', 'request_id': 'req-5bae74f3-c848-43bf-babd-1cac8088921f'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:28.638165 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-5bae74f3-c848-43bf-babd-1cac8088921f admin admin] The token cache is being invalidate because project 9cc67c4674b64223ae2a8c9e7de9b6d4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80298) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 12:21:28.638595 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-5bae74f3-c848-43bf-babd-1cac8088921f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:28.639389 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-5bae74f3-c848-43bf-babd-1cac8088921f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:28.641312 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1283/2567] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Jul 25 12:21:28 2022] DELETE /identity/v3/projects/9cc67c4674b64223ae2a8c9e7de9b6d4 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:21:28.649795 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057f21a6-b497-4fa6-bcd2-4fc748bbfc09 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:28.650394 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057f21a6-b497-4fa6-bcd2-4fc748bbfc09 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:28.651056 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057f21a6-b497-4fa6-bcd2-4fc748bbfc09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:28.673393 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-057f21a6-b497-4fa6-bcd2-4fc748bbfc09 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:21:28.674608 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-057f21a6-b497-4fa6-bcd2-4fc748bbfc09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:21:28.745263 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1285/2568] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:21:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:21:28.753955 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0434a9ad-673b-4f78-99ea-819fc8190b51 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:28.754382 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0434a9ad-673b-4f78-99ea-819fc8190b51 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:28.754715 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0434a9ad-673b-4f78-99ea-819fc8190b51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:28.796152 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-0434a9ad-673b-4f78-99ea-819fc8190b51 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:21:28.797573 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-0434a9ad-673b-4f78-99ea-819fc8190b51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:21:28.806725 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1284/2569] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:21:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:21:28.810699 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057f21a6-b497-4fa6-bcd2-4fc748bbfc09 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:21:28.821823 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057f21a6-b497-4fa6-bcd2-4fc748bbfc09 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:21:28.827504 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293b7e8a-678e-4d27-8aa4-ab79f1642450 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:21:28.828307 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293b7e8a-678e-4d27-8aa4-ab79f1642450 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:28.828627 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293b7e8a-678e-4d27-8aa4-ab79f1642450 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:28.828866 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293b7e8a-678e-4d27-8aa4-ab79f1642450 admin admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:28.829769 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293b7e8a-678e-4d27-8aa4-ab79f1642450 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:21:28.830094 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293b7e8a-678e-4d27-8aa4-ab79f1642450 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:21:28.831748 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293b7e8a-678e-4d27-8aa4-ab79f1642450 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:21:28.845494 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1286/2570] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Jul 25 12:21:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 12:21:28.849917 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0434a9ad-673b-4f78-99ea-819fc8190b51 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:21:28.859997 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0434a9ad-673b-4f78-99ea-819fc8190b51 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:21:28.867134 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a59df78-0364-4816-9e65-682fc055f45b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:21:28.868222 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a59df78-0364-4816-9e65-682fc055f45b admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:28.868716 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a59df78-0364-4816-9e65-682fc055f45b admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:28.869149 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a59df78-0364-4816-9e65-682fc055f45b admin admin] PATH_INFO: `/v3/projects` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:28.869958 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a59df78-0364-4816-9e65-682fc055f45b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:21:28.872494 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a59df78-0364-4816-9e65-682fc055f45b admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:21:28.922241 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1285/2571] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Jul 25 12:21:28 2022] POST /identity/v3/projects => generated 373 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:21:28.926085 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293b7e8a-678e-4d27-8aa4-ab79f1642450 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:21:28.947154 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293b7e8a-678e-4d27-8aa4-ab79f1642450 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:21:29.024687 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10c4b61-53d8-4efe-9309-650496f8ea19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:21:29.025516 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10c4b61-53d8-4efe-9309-650496f8ea19 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:29.025879 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10c4b61-53d8-4efe-9309-650496f8ea19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:29.026233 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10c4b61-53d8-4efe-9309-650496f8ea19 admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:29.027007 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10c4b61-53d8-4efe-9309-650496f8ea19 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:21:29.029039 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10c4b61-53d8-4efe-9309-650496f8ea19 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:21:29.058513 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1287/2572] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Jul 25 12:21:28 2022] POST /identity/v3/users => generated 287 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:21:29.062537 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a59df78-0364-4816-9e65-682fc055f45b admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:21:29.074866 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a59df78-0364-4816-9e65-682fc055f45b admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:21:29.080031 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68668e93-3972-46fa-b540-d5b9bc00f4cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:21:29.080699 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68668e93-3972-46fa-b540-d5b9bc00f4cd admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:29.080982 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68668e93-3972-46fa-b540-d5b9bc00f4cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:29.081276 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68668e93-3972-46fa-b540-d5b9bc00f4cd admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:29.081999 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68668e93-3972-46fa-b540-d5b9bc00f4cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:21:29.083928 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68668e93-3972-46fa-b540-d5b9bc00f4cd admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:21:29.094410 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1286/2573] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Jul 25 12:21:29 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:21:29.098396 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10c4b61-53d8-4efe-9309-650496f8ea19 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:21:29.109103 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10c4b61-53d8-4efe-9309-650496f8ea19 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:21:29.114061 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e890f4c7-d465-48f5-ae32-02a10a6d996f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:21:29.114735 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e890f4c7-d465-48f5-ae32-02a10a6d996f admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:29.115023 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e890f4c7-d465-48f5-ae32-02a10a6d996f admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:29.115307 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e890f4c7-d465-48f5-ae32-02a10a6d996f admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:29.116074 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e890f4c7-d465-48f5-ae32-02a10a6d996f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:21:29.118190 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e890f4c7-d465-48f5-ae32-02a10a6d996f admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:21:29.129255 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1288/2574] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Jul 25 12:21:29 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 12:21:29.134190 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68668e93-3972-46fa-b540-d5b9bc00f4cd admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:21:29.144218 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68668e93-3972-46fa-b540-d5b9bc00f4cd admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:21:29.148990 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b848bcd7-2d8a-4b27-90ea-484bb7137781 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:21:29.154026 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b848bcd7-2d8a-4b27-90ea-484bb7137781 admin admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:29.154320 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b848bcd7-2d8a-4b27-90ea-484bb7137781 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:29.154616 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b848bcd7-2d8a-4b27-90ea-484bb7137781 admin admin] PATH_INFO: `/v3/projects/e9eaa14479b64763a86720ccf40e395a/users/4ef93840ca1e499ab87431dcba856308/roles/472313ae69174578824df0bed6390744` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:29.170307 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b848bcd7-2d8a-4b27-90ea-484bb7137781 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e9eaa14479b64763a86720ccf40e395a, user_id=4ef93840ca1e499ab87431dcba856308, role_id=472313ae69174578824df0bed6390744)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:21:29.173024 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b848bcd7-2d8a-4b27-90ea-484bb7137781 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:21:29.186416 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1287/2575] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Jul 25 12:21:29 2022] PUT /identity/v3/projects/e9eaa14479b64763a86720ccf40e395a/users/4ef93840ca1e499ab87431dcba856308/roles/472313ae69174578824df0bed6390744 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 12:21:29.191536 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13932237-1f84-405e-90a9-040009e7fe63 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:29.191853 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13932237-1f84-405e-90a9-040009e7fe63 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:29.192135 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13932237-1f84-405e-90a9-040009e7fe63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:29.212605 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-13932237-1f84-405e-90a9-040009e7fe63 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:21:29.213809 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-13932237-1f84-405e-90a9-040009e7fe63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:21:29.231618 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:21:29.276081 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1289/2576] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:21:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:21:29.286063 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b848bcd7-2d8a-4b27-90ea-484bb7137781 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:21:29.287879 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-b848bcd7-2d8a-4b27-90ea-484bb7137781 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:21:29.344725 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b848bcd7-2d8a-4b27-90ea-484bb7137781 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:21:29.348979 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afac74a5-e28e-41fe-9610-7c14721afe7c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51733781201243e8abf9eca39b207750', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:21:29.349636 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afac74a5-e28e-41fe-9610-7c14721afe7c service neutron] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:29.349925 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afac74a5-e28e-41fe-9610-7c14721afe7c service neutron] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:29.350208 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afac74a5-e28e-41fe-9610-7c14721afe7c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:29.358178 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:21:29.360603 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afac74a5-e28e-41fe-9610-7c14721afe7c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:21:29.362870 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afac74a5-e28e-41fe-9610-7c14721afe7c service neutron] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:21:29.377195 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1288/2577] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:21:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 12:21:30.019257 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1eb6126-fe65-485d-b517-5457a2e3b406 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:30.019766 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1eb6126-fe65-485d-b517-5457a2e3b406 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:30.019852 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1eb6126-fe65-485d-b517-5457a2e3b406 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:30.050560 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-a1eb6126-fe65-485d-b517-5457a2e3b406 None None] MFA Rules not processed for user `4ef93840ca1e499ab87431dcba856308`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:21:30.051838 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-a1eb6126-fe65-485d-b517-5457a2e3b406 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:21:30.169572 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1290/2578] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:21:30 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:21:30.173733 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afac74a5-e28e-41fe-9610-7c14721afe7c service neutron] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:21:30.185052 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afac74a5-e28e-41fe-9610-7c14721afe7c service neutron] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:21:30.191286 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bfc315-3de7-434e-9d8f-f382224a0bac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:21:30.192117 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bfc315-3de7-434e-9d8f-f382224a0bac admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:30.192970 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bfc315-3de7-434e-9d8f-f382224a0bac admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:30.192970 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bfc315-3de7-434e-9d8f-f382224a0bac admin admin] PATH_INFO: `/v3/projects` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:30.193790 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72bfc315-3de7-434e-9d8f-f382224a0bac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:21:30.196023 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72bfc315-3de7-434e-9d8f-f382224a0bac admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:21:30.243622 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1289/2579] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Jul 25 12:21:30 2022] POST /identity/v3/projects => generated 373 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:21:30.245622 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1eb6126-fe65-485d-b517-5457a2e3b406 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:21:30.260141 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1eb6126-fe65-485d-b517-5457a2e3b406 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:21:30.319162 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e48777-9192-41d7-af23-fc4f3f7546e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:21:30.319934 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e48777-9192-41d7-af23-fc4f3f7546e1 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:30.320229 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e48777-9192-41d7-af23-fc4f3f7546e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:30.320518 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e48777-9192-41d7-af23-fc4f3f7546e1 admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:30.321232 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e48777-9192-41d7-af23-fc4f3f7546e1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:21:30.323155 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e48777-9192-41d7-af23-fc4f3f7546e1 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:21:30.348801 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1291/2580] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Jul 25 12:21:30 2022] POST /identity/v3/users => generated 293 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 12:21:30.349963 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bfc315-3de7-434e-9d8f-f382224a0bac admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:21:30.361489 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bfc315-3de7-434e-9d8f-f382224a0bac admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:21:30.366083 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8756ed8f-daab-44a0-8ef1-fb3758d9e501 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:21:30.367645 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8756ed8f-daab-44a0-8ef1-fb3758d9e501 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:30.367947 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8756ed8f-daab-44a0-8ef1-fb3758d9e501 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:30.368238 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8756ed8f-daab-44a0-8ef1-fb3758d9e501 admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:30.369006 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8756ed8f-daab-44a0-8ef1-fb3758d9e501 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:21:30.370610 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8756ed8f-daab-44a0-8ef1-fb3758d9e501 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:21:30.381329 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1290/2581] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Jul 25 12:21:30 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 12:21:30.385699 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e48777-9192-41d7-af23-fc4f3f7546e1 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:21:30.395728 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e48777-9192-41d7-af23-fc4f3f7546e1 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:21:30.401340 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218aed75-2eda-4595-a5b6-40967919e1fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:21:30.402227 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218aed75-2eda-4595-a5b6-40967919e1fc admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:30.402514 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218aed75-2eda-4595-a5b6-40967919e1fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:30.402796 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218aed75-2eda-4595-a5b6-40967919e1fc admin admin] PATH_INFO: `/v3/projects/cc5f40813e434b6ebbf6908896b992f0/users/c41698cb7564488595cc9d70a23312b7/roles/a7d20885b6d94928a34b3cc5b3a00006` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:30.418231 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218aed75-2eda-4595-a5b6-40967919e1fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=cc5f40813e434b6ebbf6908896b992f0, user_id=c41698cb7564488595cc9d70a23312b7, role_id=a7d20885b6d94928a34b3cc5b3a00006)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:21:30.420945 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218aed75-2eda-4595-a5b6-40967919e1fc admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:21:30.433504 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1292/2582] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Jul 25 12:21:30 2022] PUT /identity/v3/projects/cc5f40813e434b6ebbf6908896b992f0/users/c41698cb7564488595cc9d70a23312b7/roles/a7d20885b6d94928a34b3cc5b3a00006 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:21:30.437437 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8756ed8f-daab-44a0-8ef1-fb3758d9e501 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:21:30.447503 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8756ed8f-daab-44a0-8ef1-fb3758d9e501 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:21:30.504532 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e9c914-887c-4e2c-9a5a-41efcdce07f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:21:30.505204 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e9c914-887c-4e2c-9a5a-41efcdce07f6 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:30.505497 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e9c914-887c-4e2c-9a5a-41efcdce07f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:30.505781 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e9c914-887c-4e2c-9a5a-41efcdce07f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:30.506507 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e9c914-887c-4e2c-9a5a-41efcdce07f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:21:30.508779 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e9c914-887c-4e2c-9a5a-41efcdce07f6 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:21:30.518405 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1291/2583] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Jul 25 12:21:30 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:21:30.522519 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218aed75-2eda-4595-a5b6-40967919e1fc admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:21:30.533010 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218aed75-2eda-4595-a5b6-40967919e1fc admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:21:30.537627 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa320851-478f-426b-8143-cc17a105a2b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:21:30.538489 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa320851-478f-426b-8143-cc17a105a2b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:30.538773 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa320851-478f-426b-8143-cc17a105a2b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:30.539047 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa320851-478f-426b-8143-cc17a105a2b9 admin admin] PATH_INFO: `/v3/projects/cc5f40813e434b6ebbf6908896b992f0/users/c41698cb7564488595cc9d70a23312b7/roles/a7d20885b6d94928a34b3cc5b3a00006` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:30.541048 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa320851-478f-426b-8143-cc17a105a2b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cc5f40813e434b6ebbf6908896b992f0, user_id=c41698cb7564488595cc9d70a23312b7, role_id=a7d20885b6d94928a34b3cc5b3a00006)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:21:30.543683 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa320851-478f-426b-8143-cc17a105a2b9 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:21:30.554082 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1293/2584] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Jul 25 12:21:30 2022] PUT /identity/v3/projects/cc5f40813e434b6ebbf6908896b992f0/users/c41698cb7564488595cc9d70a23312b7/roles/a7d20885b6d94928a34b3cc5b3a00006 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:21:31.151781 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94367a73-f7cb-4f59-a131-35ef3fb459fe None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:31.152213 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94367a73-f7cb-4f59-a131-35ef3fb459fe None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:31.152804 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94367a73-f7cb-4f59-a131-35ef3fb459fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:31.184592 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-94367a73-f7cb-4f59-a131-35ef3fb459fe None None] MFA Rules not processed for user `c41698cb7564488595cc9d70a23312b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:21:31.185950 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-94367a73-f7cb-4f59-a131-35ef3fb459fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:21:31.272130 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1292/2585] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:21:31 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:21:31.284303 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa320851-478f-426b-8143-cc17a105a2b9 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:21:31.286039 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-aa320851-478f-426b-8143-cc17a105a2b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:21:31.296364 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:21:31.358140 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa320851-478f-426b-8143-cc17a105a2b9 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:21:31.361718 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13737913-025b-4d60-9fa6-9a7913eba649 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f89ce6f0cf04373a414d927fa0f8a7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:21:31.362361 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13737913-025b-4d60-9fa6-9a7913eba649 service cinder] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:31.362648 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13737913-025b-4d60-9fa6-9a7913eba649 service cinder] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:31.362930 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13737913-025b-4d60-9fa6-9a7913eba649 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:31.418612 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13737913-025b-4d60-9fa6-9a7913eba649 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:21:31.420104 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13737913-025b-4d60-9fa6-9a7913eba649 service cinder] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:21:31.428964 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1294/2586] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:21:31 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 12:21:35.933398 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94367a73-f7cb-4f59-a131-35ef3fb459fe None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:21:35.943185 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94367a73-f7cb-4f59-a131-35ef3fb459fe None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:21:35.946910 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bea4507-d1e9-4dd7-b8ba-f2b49cb3e5a2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f89ce6f0cf04373a414d927fa0f8a7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:21:35.947579 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bea4507-d1e9-4dd7-b8ba-f2b49cb3e5a2 service cinder] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:35.947876 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bea4507-d1e9-4dd7-b8ba-f2b49cb3e5a2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:35.948160 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bea4507-d1e9-4dd7-b8ba-f2b49cb3e5a2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:35.957224 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bea4507-d1e9-4dd7-b8ba-f2b49cb3e5a2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:21:35.958653 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bea4507-d1e9-4dd7-b8ba-f2b49cb3e5a2 service cinder] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:21:35.968635 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1293/2587] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:21:35 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:21:38.069026 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13737913-025b-4d60-9fa6-9a7913eba649 service cinder] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:21:38.070435 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-13737913-025b-4d60-9fa6-9a7913eba649 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:21:38.135574 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13737913-025b-4d60-9fa6-9a7913eba649 service cinder] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:21:38.140078 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a71cca-42b2-46d1-8d7e-0400ac2b0fa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:21:38.140945 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a71cca-42b2-46d1-8d7e-0400ac2b0fa5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:38.141234 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a71cca-42b2-46d1-8d7e-0400ac2b0fa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:38.141518 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a71cca-42b2-46d1-8d7e-0400ac2b0fa5 admin admin] PATH_INFO: `/v3/users/89aa9d11a1d644a180916cefbbe8dbf6` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:38.142765 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a71cca-42b2-46d1-8d7e-0400ac2b0fa5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=89aa9d11a1d644a180916cefbbe8dbf6)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:21:38.144753 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a71cca-42b2-46d1-8d7e-0400ac2b0fa5 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:21:38.191778 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-17a71cca-42b2-46d1-8d7e-0400ac2b0fa5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '89aa9d11a1d644a180916cefbbe8dbf6'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:38.198498 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-17a71cca-42b2-46d1-8d7e-0400ac2b0fa5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '89aa9d11a1d644a180916cefbbe8dbf6'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:38.205551 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-17a71cca-42b2-46d1-8d7e-0400ac2b0fa5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '89aa9d11a1d644a180916cefbbe8dbf6'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:38.206684 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:21:38.220406 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-17a71cca-42b2-46d1-8d7e-0400ac2b0fa5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '89aa9d11a1d644a180916cefbbe8dbf6'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:38.221044 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-17a71cca-42b2-46d1-8d7e-0400ac2b0fa5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '89aa9d11a1d644a180916cefbbe8dbf6'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:38.227504 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1295/2588] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Jul 25 12:21:38 2022] DELETE /identity/v3/users/89aa9d11a1d644a180916cefbbe8dbf6 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:21:38.232003 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb737241-ba75-4293-bdda-eeba25eaf8ef None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:38.232459 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb737241-ba75-4293-bdda-eeba25eaf8ef None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:38.232574 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb737241-ba75-4293-bdda-eeba25eaf8ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:38.250682 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-cb737241-ba75-4293-bdda-eeba25eaf8ef None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:21:38.251911 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-cb737241-ba75-4293-bdda-eeba25eaf8ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:21:38.278369 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:21:38.312682 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1294/2589] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:21:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:21:38.322908 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a71cca-42b2-46d1-8d7e-0400ac2b0fa5 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:21:38.324307 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-17a71cca-42b2-46d1-8d7e-0400ac2b0fa5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:21:38.379595 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a71cca-42b2-46d1-8d7e-0400ac2b0fa5 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:21:38.383752 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad931a5-6b81-4082-9c43-d189746c0282 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51733781201243e8abf9eca39b207750', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:21:38.384388 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad931a5-6b81-4082-9c43-d189746c0282 service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:38.384678 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad931a5-6b81-4082-9c43-d189746c0282 service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:38.384959 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad931a5-6b81-4082-9c43-d189746c0282 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:38.393706 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad931a5-6b81-4082-9c43-d189746c0282 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:21:38.395115 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad931a5-6b81-4082-9c43-d189746c0282 service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:21:38.405353 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1296/2590] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:21:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 12:21:38.685442 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb737241-ba75-4293-bdda-eeba25eaf8ef None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:21:38.687082 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-cb737241-ba75-4293-bdda-eeba25eaf8ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:21:38.701036 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb737241-ba75-4293-bdda-eeba25eaf8ef None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:21:38.705868 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9759a4-a49a-4cff-9e5a-f2e159fc423d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:21:38.706841 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9759a4-a49a-4cff-9e5a-f2e159fc423d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:38.707276 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9759a4-a49a-4cff-9e5a-f2e159fc423d admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:38.707528 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9759a4-a49a-4cff-9e5a-f2e159fc423d admin admin] PATH_INFO: `/v3/projects/7b3f8796504c47fab663669f741348dc` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:38.708841 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9759a4-a49a-4cff-9e5a-f2e159fc423d admin admin] RBAC: Authorizing `identity:delete_project(project_id=7b3f8796504c47fab663669f741348dc)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:21:38.710807 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9759a4-a49a-4cff-9e5a-f2e159fc423d admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:21:38.770424 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-7e9759a4-a49a-4cff-9e5a-f2e159fc423d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7b3f8796504c47fab663669f741348dc', 'request_id': 'req-7e9759a4-a49a-4cff-9e5a-f2e159fc423d'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:38.784521 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-7e9759a4-a49a-4cff-9e5a-f2e159fc423d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7b3f8796504c47fab663669f741348dc', 'request_id': 'req-7e9759a4-a49a-4cff-9e5a-f2e159fc423d'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:38.799779 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-7e9759a4-a49a-4cff-9e5a-f2e159fc423d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7b3f8796504c47fab663669f741348dc', 'request_id': 'req-7e9759a4-a49a-4cff-9e5a-f2e159fc423d'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:38.814714 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-7e9759a4-a49a-4cff-9e5a-f2e159fc423d admin admin] The token cache is being invalidate because project 7b3f8796504c47fab663669f741348dc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80298) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 12:21:38.815035 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-7e9759a4-a49a-4cff-9e5a-f2e159fc423d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:38.817106 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-7e9759a4-a49a-4cff-9e5a-f2e159fc423d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:38.817534 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1295/2591] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Jul 25 12:21:38 2022] DELETE /identity/v3/projects/7b3f8796504c47fab663669f741348dc => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:21:38.822200 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad931a5-6b81-4082-9c43-d189746c0282 service neutron] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:21:38.832261 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad931a5-6b81-4082-9c43-d189746c0282 service neutron] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:21:38.896612 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd4fbfd-b06e-42af-b671-f666e5eb064a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:21:38.898364 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd4fbfd-b06e-42af-b671-f666e5eb064a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:38.898688 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd4fbfd-b06e-42af-b671-f666e5eb064a admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:38.899575 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd4fbfd-b06e-42af-b671-f666e5eb064a admin admin] PATH_INFO: `/v3/users/dc447433b7084f5c9b259a02637aab51` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:38.900715 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd4fbfd-b06e-42af-b671-f666e5eb064a admin admin] RBAC: Authorizing `identity:delete_user(user_id=dc447433b7084f5c9b259a02637aab51)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:21:38.903213 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd4fbfd-b06e-42af-b671-f666e5eb064a admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:21:38.950733 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-3fd4fbfd-b06e-42af-b671-f666e5eb064a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dc447433b7084f5c9b259a02637aab51'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:38.958097 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-3fd4fbfd-b06e-42af-b671-f666e5eb064a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dc447433b7084f5c9b259a02637aab51'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:38.965497 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-3fd4fbfd-b06e-42af-b671-f666e5eb064a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dc447433b7084f5c9b259a02637aab51'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:38.977332 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-3fd4fbfd-b06e-42af-b671-f666e5eb064a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dc447433b7084f5c9b259a02637aab51'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:38.977955 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-3fd4fbfd-b06e-42af-b671-f666e5eb064a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dc447433b7084f5c9b259a02637aab51'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:38.983752 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1297/2592] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Jul 25 12:21:38 2022] DELETE /identity/v3/users/dc447433b7084f5c9b259a02637aab51 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 12:21:39.255203 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9759a4-a49a-4cff-9e5a-f2e159fc423d admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:21:39.257329 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-7e9759a4-a49a-4cff-9e5a-f2e159fc423d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:21:39.278705 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9759a4-a49a-4cff-9e5a-f2e159fc423d admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:21:39.336388 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a120fbcb-b9ad-4181-93f3-ad280ab9ea07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:21:39.337241 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a120fbcb-b9ad-4181-93f3-ad280ab9ea07 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:39.337524 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a120fbcb-b9ad-4181-93f3-ad280ab9ea07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:39.337804 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a120fbcb-b9ad-4181-93f3-ad280ab9ea07 admin admin] PATH_INFO: `/v3/projects/140ca86474e84c05a2b683060f71837c` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:39.338981 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a120fbcb-b9ad-4181-93f3-ad280ab9ea07 admin admin] RBAC: Authorizing `identity:delete_project(project_id=140ca86474e84c05a2b683060f71837c)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:21:39.340930 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a120fbcb-b9ad-4181-93f3-ad280ab9ea07 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:21:39.387179 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-a120fbcb-b9ad-4181-93f3-ad280ab9ea07 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '140ca86474e84c05a2b683060f71837c', 'request_id': 'req-a120fbcb-b9ad-4181-93f3-ad280ab9ea07'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:39.394225 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-a120fbcb-b9ad-4181-93f3-ad280ab9ea07 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '140ca86474e84c05a2b683060f71837c', 'request_id': 'req-a120fbcb-b9ad-4181-93f3-ad280ab9ea07'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:39.395456 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:21:39.401872 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-a120fbcb-b9ad-4181-93f3-ad280ab9ea07 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '140ca86474e84c05a2b683060f71837c', 'request_id': 'req-a120fbcb-b9ad-4181-93f3-ad280ab9ea07'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:39.414303 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-a120fbcb-b9ad-4181-93f3-ad280ab9ea07 admin admin] The token cache is being invalidate because project 140ca86474e84c05a2b683060f71837c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80298) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 12:21:39.414592 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-a120fbcb-b9ad-4181-93f3-ad280ab9ea07 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:39.415222 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-a120fbcb-b9ad-4181-93f3-ad280ab9ea07 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:39.416756 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1296/2593] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Jul 25 12:21:39 2022] DELETE /identity/v3/projects/140ca86474e84c05a2b683060f71837c => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:21:39.421666 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25eaed90-d72d-4391-ada5-2e56723bffde None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:39.422126 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25eaed90-d72d-4391-ada5-2e56723bffde None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:39.422250 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25eaed90-d72d-4391-ada5-2e56723bffde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:39.430680 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:21:39.446077 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-25eaed90-d72d-4391-ada5-2e56723bffde None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:21:39.447299 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-25eaed90-d72d-4391-ada5-2e56723bffde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:21:39.505108 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1298/2594] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:21:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:21:39.513720 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51df7648-b1bd-4633-ad80-8784fce6d413 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:39.514018 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51df7648-b1bd-4633-ad80-8784fce6d413 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:39.514300 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51df7648-b1bd-4633-ad80-8784fce6d413 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:39.540777 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-51df7648-b1bd-4633-ad80-8784fce6d413 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:21:39.542016 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-51df7648-b1bd-4633-ad80-8784fce6d413 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:21:39.550963 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1297/2595] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:21:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:21:39.555749 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25eaed90-d72d-4391-ada5-2e56723bffde None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:21:39.565788 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25eaed90-d72d-4391-ada5-2e56723bffde None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:21:39.571143 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62132f17-ab9c-4685-8779-c2e6607bbc5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:21:39.571828 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62132f17-ab9c-4685-8779-c2e6607bbc5b admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:39.572117 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62132f17-ab9c-4685-8779-c2e6607bbc5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:39.572400 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62132f17-ab9c-4685-8779-c2e6607bbc5b admin admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:39.573146 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62132f17-ab9c-4685-8779-c2e6607bbc5b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:21:39.573441 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62132f17-ab9c-4685-8779-c2e6607bbc5b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:21:39.575032 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62132f17-ab9c-4685-8779-c2e6607bbc5b admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:21:39.586644 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1299/2596] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Jul 25 12:21:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:21:39.590712 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51df7648-b1bd-4633-ad80-8784fce6d413 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:21:39.600770 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51df7648-b1bd-4633-ad80-8784fce6d413 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:21:39.605525 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962d20ff-16c3-4100-98d1-a2dee7197cb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:21:39.606177 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962d20ff-16c3-4100-98d1-a2dee7197cb6 admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:39.606467 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962d20ff-16c3-4100-98d1-a2dee7197cb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:39.606747 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962d20ff-16c3-4100-98d1-a2dee7197cb6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:39.607475 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962d20ff-16c3-4100-98d1-a2dee7197cb6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:21:39.609409 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962d20ff-16c3-4100-98d1-a2dee7197cb6 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:21:39.637557 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1298/2597] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Jul 25 12:21:39 2022] POST /identity/v3/projects => generated 375 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:21:39.641905 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62132f17-ab9c-4685-8779-c2e6607bbc5b admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:21:39.652374 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62132f17-ab9c-4685-8779-c2e6607bbc5b admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:21:39.707971 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c135c84b-5410-4d8e-b703-e81593e6ac61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:21:39.708637 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c135c84b-5410-4d8e-b703-e81593e6ac61 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:39.708949 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c135c84b-5410-4d8e-b703-e81593e6ac61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:39.709254 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c135c84b-5410-4d8e-b703-e81593e6ac61 admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:39.709943 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c135c84b-5410-4d8e-b703-e81593e6ac61 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:21:39.711905 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c135c84b-5410-4d8e-b703-e81593e6ac61 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:21:39.734638 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1300/2598] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Jul 25 12:21:39 2022] POST /identity/v3/users => generated 288 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:21:39.736351 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962d20ff-16c3-4100-98d1-a2dee7197cb6 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:21:39.747125 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962d20ff-16c3-4100-98d1-a2dee7197cb6 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:21:39.751764 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdb29fd-391c-4114-806b-b57146772db4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:21:39.752441 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdb29fd-391c-4114-806b-b57146772db4 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:39.752715 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdb29fd-391c-4114-806b-b57146772db4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:39.753000 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdb29fd-391c-4114-806b-b57146772db4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:39.753734 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecdb29fd-391c-4114-806b-b57146772db4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:21:39.755486 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecdb29fd-391c-4114-806b-b57146772db4 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:21:39.766207 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1299/2599] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Jul 25 12:21:39 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:21:39.770224 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c135c84b-5410-4d8e-b703-e81593e6ac61 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:21:39.784427 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c135c84b-5410-4d8e-b703-e81593e6ac61 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:21:39.789698 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1f05d8-facf-4546-95aa-70df3e4d03b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:21:39.790363 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1f05d8-facf-4546-95aa-70df3e4d03b9 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:39.790658 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1f05d8-facf-4546-95aa-70df3e4d03b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:39.790938 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1f05d8-facf-4546-95aa-70df3e4d03b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:39.791853 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1f05d8-facf-4546-95aa-70df3e4d03b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:21:39.793429 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1f05d8-facf-4546-95aa-70df3e4d03b9 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:21:39.807022 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1301/2600] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Jul 25 12:21:39 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:21:39.810963 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdb29fd-391c-4114-806b-b57146772db4 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:21:39.821302 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdb29fd-391c-4114-806b-b57146772db4 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:21:39.825944 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d90c24-0c51-42b7-aa71-13efb319ffce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:21:39.826825 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d90c24-0c51-42b7-aa71-13efb319ffce admin admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:39.827118 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d90c24-0c51-42b7-aa71-13efb319ffce admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:39.827400 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d90c24-0c51-42b7-aa71-13efb319ffce admin admin] PATH_INFO: `/v3/projects/d8529c14d4164488b57600795f3ac919/users/504aef7b26144da3953fad61d759894b/roles/472313ae69174578824df0bed6390744` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:39.842981 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d90c24-0c51-42b7-aa71-13efb319ffce admin admin] RBAC: Authorizing `identity:create_grant(project_id=d8529c14d4164488b57600795f3ac919, user_id=504aef7b26144da3953fad61d759894b, role_id=472313ae69174578824df0bed6390744)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:21:39.845715 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d90c24-0c51-42b7-aa71-13efb319ffce admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:21:39.860329 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1300/2601] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Jul 25 12:21:39 2022] PUT /identity/v3/projects/d8529c14d4164488b57600795f3ac919/users/504aef7b26144da3953fad61d759894b/roles/472313ae69174578824df0bed6390744 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:21:39.871161 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868fcf93-b82b-47ee-a0cc-9b1bec8e89d3 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:39.871452 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868fcf93-b82b-47ee-a0cc-9b1bec8e89d3 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:39.872065 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868fcf93-b82b-47ee-a0cc-9b1bec8e89d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:39.901127 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-868fcf93-b82b-47ee-a0cc-9b1bec8e89d3 None None] MFA Rules not processed for user `504aef7b26144da3953fad61d759894b`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:21:39.902348 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-868fcf93-b82b-47ee-a0cc-9b1bec8e89d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:21:39.977236 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:21:39.986751 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1302/2602] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:21:39 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 12:21:39.987413 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d90c24-0c51-42b7-aa71-13efb319ffce admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:21:40.097999 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d90c24-0c51-42b7-aa71-13efb319ffce admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:21:40.103719 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9eb138e-50be-476e-b30e-a5e06e460d26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:21:40.104475 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9eb138e-50be-476e-b30e-a5e06e460d26 admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:40.104840 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9eb138e-50be-476e-b30e-a5e06e460d26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:40.105205 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9eb138e-50be-476e-b30e-a5e06e460d26 admin admin] PATH_INFO: `/v3/projects` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:40.105981 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9eb138e-50be-476e-b30e-a5e06e460d26 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:21:40.107992 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9eb138e-50be-476e-b30e-a5e06e460d26 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:21:40.135347 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1301/2603] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Jul 25 12:21:39 2022] POST /identity/v3/projects => generated 377 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 12:21:40.139457 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868fcf93-b82b-47ee-a0cc-9b1bec8e89d3 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:21:40.150892 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868fcf93-b82b-47ee-a0cc-9b1bec8e89d3 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:21:40.205883 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b571dc73-181e-4728-a2c5-d268b3711e9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:21:40.206551 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b571dc73-181e-4728-a2c5-d268b3711e9d admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:40.206839 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b571dc73-181e-4728-a2c5-d268b3711e9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:40.207120 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b571dc73-181e-4728-a2c5-d268b3711e9d admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:40.207864 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b571dc73-181e-4728-a2c5-d268b3711e9d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:21:40.210316 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b571dc73-181e-4728-a2c5-d268b3711e9d admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:21:40.232157 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1303/2604] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Jul 25 12:21:40 2022] POST /identity/v3/users => generated 289 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:21:40.235999 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9eb138e-50be-476e-b30e-a5e06e460d26 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:21:40.246234 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9eb138e-50be-476e-b30e-a5e06e460d26 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:21:40.252339 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceaa3f8a-20d7-4f5d-91ea-f03ce5ae264c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:21:40.253433 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceaa3f8a-20d7-4f5d-91ea-f03ce5ae264c admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:40.253939 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceaa3f8a-20d7-4f5d-91ea-f03ce5ae264c admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:40.254434 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceaa3f8a-20d7-4f5d-91ea-f03ce5ae264c admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:40.255760 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceaa3f8a-20d7-4f5d-91ea-f03ce5ae264c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:21:40.257644 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceaa3f8a-20d7-4f5d-91ea-f03ce5ae264c admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:21:40.268578 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1302/2605] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Jul 25 12:21:40 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:21:40.273326 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b571dc73-181e-4728-a2c5-d268b3711e9d admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:21:40.284720 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b571dc73-181e-4728-a2c5-d268b3711e9d admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:21:40.290226 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db85f18-2f94-41a7-be01-fe62089c50e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:21:40.290933 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db85f18-2f94-41a7-be01-fe62089c50e6 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:40.291225 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db85f18-2f94-41a7-be01-fe62089c50e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:40.291511 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db85f18-2f94-41a7-be01-fe62089c50e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:40.292288 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db85f18-2f94-41a7-be01-fe62089c50e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:21:40.293981 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db85f18-2f94-41a7-be01-fe62089c50e6 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:21:40.304714 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1304/2606] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Jul 25 12:21:40 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:21:40.308892 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceaa3f8a-20d7-4f5d-91ea-f03ce5ae264c admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:21:40.318839 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceaa3f8a-20d7-4f5d-91ea-f03ce5ae264c admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:21:40.323390 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff85717-84c9-4f29-9619-ab0742576c49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:21:40.324307 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff85717-84c9-4f29-9619-ab0742576c49 admin admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:40.324592 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff85717-84c9-4f29-9619-ab0742576c49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:40.324874 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff85717-84c9-4f29-9619-ab0742576c49 admin admin] PATH_INFO: `/v3/projects/43328f65d95a45f383b61da6af7b4639/users/abf3ff102f6b4f9182aeaa4767ff1022/roles/472313ae69174578824df0bed6390744` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:40.340198 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff85717-84c9-4f29-9619-ab0742576c49 admin admin] RBAC: Authorizing `identity:create_grant(project_id=43328f65d95a45f383b61da6af7b4639, user_id=abf3ff102f6b4f9182aeaa4767ff1022, role_id=472313ae69174578824df0bed6390744)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:21:40.342936 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff85717-84c9-4f29-9619-ab0742576c49 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:21:40.356454 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1303/2607] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Jul 25 12:21:40 2022] PUT /identity/v3/projects/43328f65d95a45f383b61da6af7b4639/users/abf3ff102f6b4f9182aeaa4767ff1022/roles/472313ae69174578824df0bed6390744 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 12:21:40.423189 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b93a80-a1c5-4427-b79b-a1c3eddca5d7 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:40.423763 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b93a80-a1c5-4427-b79b-a1c3eddca5d7 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:40.423894 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b93a80-a1c5-4427-b79b-a1c3eddca5d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:40.451639 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-56b93a80-a1c5-4427-b79b-a1c3eddca5d7 None None] MFA Rules not processed for user `abf3ff102f6b4f9182aeaa4767ff1022`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:21:40.452861 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-56b93a80-a1c5-4427-b79b-a1c3eddca5d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:21:40.516111 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1305/2608] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:21:40 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:21:40.520691 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff85717-84c9-4f29-9619-ab0742576c49 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:21:40.554999 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:DELETE=2,keystone:SELECT=94,keystone:INSERT=4 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:21:40.587972 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff85717-84c9-4f29-9619-ab0742576c49 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:21:40.592855 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d689e2-78c8-4073-b9db-2fcb7e799a38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:21:40.593521 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d689e2-78c8-4073-b9db-2fcb7e799a38 admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:40.593809 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d689e2-78c8-4073-b9db-2fcb7e799a38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:40.594089 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d689e2-78c8-4073-b9db-2fcb7e799a38 admin admin] PATH_INFO: `/v3/projects` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:40.594786 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d689e2-78c8-4073-b9db-2fcb7e799a38 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:21:40.596767 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d689e2-78c8-4073-b9db-2fcb7e799a38 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:21:40.624002 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1304/2609] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Jul 25 12:21:40 2022] POST /identity/v3/projects => generated 377 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:21:40.628133 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b93a80-a1c5-4427-b79b-a1c3eddca5d7 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:21:40.639477 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b93a80-a1c5-4427-b79b-a1c3eddca5d7 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:21:40.695411 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73a8a21-ae30-41c8-9b9d-c62bfa150130 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:21:40.696121 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73a8a21-ae30-41c8-9b9d-c62bfa150130 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:40.696396 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73a8a21-ae30-41c8-9b9d-c62bfa150130 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:40.696680 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73a8a21-ae30-41c8-9b9d-c62bfa150130 admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:40.697366 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73a8a21-ae30-41c8-9b9d-c62bfa150130 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:21:40.699274 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73a8a21-ae30-41c8-9b9d-c62bfa150130 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:21:40.710044 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:21:40.723732 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1306/2610] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Jul 25 12:21:40 2022] POST /identity/v3/users => generated 295 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 12:21:40.724501 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d689e2-78c8-4073-b9db-2fcb7e799a38 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:21:40.735222 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d689e2-78c8-4073-b9db-2fcb7e799a38 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:21:40.739854 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88d943b-7e17-472f-92d0-f5b59840cae2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:21:40.741183 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88d943b-7e17-472f-92d0-f5b59840cae2 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:40.741474 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88d943b-7e17-472f-92d0-f5b59840cae2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:40.741778 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88d943b-7e17-472f-92d0-f5b59840cae2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:40.742530 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88d943b-7e17-472f-92d0-f5b59840cae2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:21:40.744246 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88d943b-7e17-472f-92d0-f5b59840cae2 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:21:40.754537 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1305/2611] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Jul 25 12:21:40 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:21:40.758418 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73a8a21-ae30-41c8-9b9d-c62bfa150130 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:21:40.768574 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73a8a21-ae30-41c8-9b9d-c62bfa150130 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:21:40.773451 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0718a43e-4154-45d5-8912-89098e66f9bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:21:40.774584 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0718a43e-4154-45d5-8912-89098e66f9bb admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:40.774826 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0718a43e-4154-45d5-8912-89098e66f9bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:40.775109 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0718a43e-4154-45d5-8912-89098e66f9bb admin admin] PATH_INFO: `/v3/projects/461a8b9163fc4912bf72fab49ea7fc22/users/c8bc7cfbb6674e24860e8f07fd2da44a/roles/a7d20885b6d94928a34b3cc5b3a00006` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:40.790720 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0718a43e-4154-45d5-8912-89098e66f9bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=461a8b9163fc4912bf72fab49ea7fc22, user_id=c8bc7cfbb6674e24860e8f07fd2da44a, role_id=a7d20885b6d94928a34b3cc5b3a00006)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:21:40.793534 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0718a43e-4154-45d5-8912-89098e66f9bb admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:21:40.807604 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1307/2612] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Jul 25 12:21:40 2022] PUT /identity/v3/projects/461a8b9163fc4912bf72fab49ea7fc22/users/c8bc7cfbb6674e24860e8f07fd2da44a/roles/a7d20885b6d94928a34b3cc5b3a00006 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 12:21:40.811667 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88d943b-7e17-472f-92d0-f5b59840cae2 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:21:40.821709 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88d943b-7e17-472f-92d0-f5b59840cae2 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:21:40.880102 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760f8037-7b93-48da-b857-1e283aa08053 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:21:40.880766 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760f8037-7b93-48da-b857-1e283aa08053 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:40.881051 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760f8037-7b93-48da-b857-1e283aa08053 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:40.881329 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760f8037-7b93-48da-b857-1e283aa08053 admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:40.882039 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760f8037-7b93-48da-b857-1e283aa08053 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:21:40.883657 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760f8037-7b93-48da-b857-1e283aa08053 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:21:40.893199 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1306/2613] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Jul 25 12:21:40 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:21:40.898149 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0718a43e-4154-45d5-8912-89098e66f9bb admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:21:40.908858 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0718a43e-4154-45d5-8912-89098e66f9bb admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:21:40.914625 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44f6950-cef5-4083-a9ea-a172279a9b12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:21:40.915665 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44f6950-cef5-4083-a9ea-a172279a9b12 admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:40.915957 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44f6950-cef5-4083-a9ea-a172279a9b12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:40.916240 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44f6950-cef5-4083-a9ea-a172279a9b12 admin admin] PATH_INFO: `/v3/projects/461a8b9163fc4912bf72fab49ea7fc22/users/c8bc7cfbb6674e24860e8f07fd2da44a/roles/a7d20885b6d94928a34b3cc5b3a00006` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:40.918245 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44f6950-cef5-4083-a9ea-a172279a9b12 admin admin] RBAC: Authorizing `identity:create_grant(project_id=461a8b9163fc4912bf72fab49ea7fc22, user_id=c8bc7cfbb6674e24860e8f07fd2da44a, role_id=a7d20885b6d94928a34b3cc5b3a00006)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:21:40.921020 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44f6950-cef5-4083-a9ea-a172279a9b12 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:21:40.931254 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1308/2614] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Jul 25 12:21:40 2022] PUT /identity/v3/projects/461a8b9163fc4912bf72fab49ea7fc22/users/c8bc7cfbb6674e24860e8f07fd2da44a/roles/a7d20885b6d94928a34b3cc5b3a00006 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:21:40.941052 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b1ecf3-ece4-41ca-b7e7-7e68e0f4c1c6 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:40.941344 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b1ecf3-ece4-41ca-b7e7-7e68e0f4c1c6 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:40.941624 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b1ecf3-ece4-41ca-b7e7-7e68e0f4c1c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:40.968993 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-d9b1ecf3-ece4-41ca-b7e7-7e68e0f4c1c6 None None] MFA Rules not processed for user `c8bc7cfbb6674e24860e8f07fd2da44a`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:21:40.970199 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-d9b1ecf3-ece4-41ca-b7e7-7e68e0f4c1c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:21:41.036183 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1307/2615] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:21:40 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:21:41.047782 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44f6950-cef5-4083-a9ea-a172279a9b12 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:21:41.049488 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-f44f6950-cef5-4083-a9ea-a172279a9b12 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:21:41.107605 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44f6950-cef5-4083-a9ea-a172279a9b12 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:21:41.111315 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017b4cb8-4c00-4aaf-9796-768c75cacd16 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f89ce6f0cf04373a414d927fa0f8a7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:21:41.112003 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017b4cb8-4c00-4aaf-9796-768c75cacd16 service cinder] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:41.112289 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017b4cb8-4c00-4aaf-9796-768c75cacd16 service cinder] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:41.112745 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017b4cb8-4c00-4aaf-9796-768c75cacd16 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:41.114084 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b1ecf3-ece4-41ca-b7e7-7e68e0f4c1c6 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:21:41.116084 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-d9b1ecf3-ece4-41ca-b7e7-7e68e0f4c1c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:21:41.150960 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:21:41.188684 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b1ecf3-ece4-41ca-b7e7-7e68e0f4c1c6 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:21:41.195255 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e9792a-3c6d-4a22-8b7f-6b1bb7497389 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:21:41.196245 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e9792a-3c6d-4a22-8b7f-6b1bb7497389 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:41.196533 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e9792a-3c6d-4a22-8b7f-6b1bb7497389 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:41.196814 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e9792a-3c6d-4a22-8b7f-6b1bb7497389 admin admin] PATH_INFO: `/v3/users/5818e45e4ba4403e9a5ea240fb5d229a` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:41.196965 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017b4cb8-4c00-4aaf-9796-768c75cacd16 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:21:41.198039 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e9792a-3c6d-4a22-8b7f-6b1bb7497389 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5818e45e4ba4403e9a5ea240fb5d229a)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:21:41.198613 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017b4cb8-4c00-4aaf-9796-768c75cacd16 service cinder] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:21:41.200301 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e9792a-3c6d-4a22-8b7f-6b1bb7497389 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:21:41.211482 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1309/2616] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:21:41 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:21:41.259349 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-90e9792a-3c6d-4a22-8b7f-6b1bb7497389 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5818e45e4ba4403e9a5ea240fb5d229a'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:41.260347 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-90e9792a-3c6d-4a22-8b7f-6b1bb7497389 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5818e45e4ba4403e9a5ea240fb5d229a'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:41.269934 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-90e9792a-3c6d-4a22-8b7f-6b1bb7497389 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5818e45e4ba4403e9a5ea240fb5d229a'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:41.291846 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-90e9792a-3c6d-4a22-8b7f-6b1bb7497389 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5818e45e4ba4403e9a5ea240fb5d229a'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:41.300313 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-90e9792a-3c6d-4a22-8b7f-6b1bb7497389 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5818e45e4ba4403e9a5ea240fb5d229a'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:41.310768 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1308/2617] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Jul 25 12:21:41 2022] DELETE /identity/v3/users/5818e45e4ba4403e9a5ea240fb5d229a => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:21:41.312402 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c36ca46-e60a-43de-8715-48bb06e40cdb None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:41.312768 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c36ca46-e60a-43de-8715-48bb06e40cdb None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:41.313122 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c36ca46-e60a-43de-8715-48bb06e40cdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:41.343279 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-7c36ca46-e60a-43de-8715-48bb06e40cdb None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:21:41.344875 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-7c36ca46-e60a-43de-8715-48bb06e40cdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:21:41.421589 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1310/2618] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:21:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 12:21:41.434631 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e9792a-3c6d-4a22-8b7f-6b1bb7497389 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:21:41.435590 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-90e9792a-3c6d-4a22-8b7f-6b1bb7497389 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:21:41.518201 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e9792a-3c6d-4a22-8b7f-6b1bb7497389 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:21:41.523623 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148adc54-7263-4704-a215-22854ee4dfb7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51733781201243e8abf9eca39b207750', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:21:41.524519 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148adc54-7263-4704-a215-22854ee4dfb7 service neutron] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:41.524842 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148adc54-7263-4704-a215-22854ee4dfb7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:41.525266 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148adc54-7263-4704-a215-22854ee4dfb7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:41.540626 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148adc54-7263-4704-a215-22854ee4dfb7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:21:41.542260 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148adc54-7263-4704-a215-22854ee4dfb7 service neutron] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:21:41.557021 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1309/2619] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:21:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 12:21:41.822995 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c36ca46-e60a-43de-8715-48bb06e40cdb None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:21:41.824638 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-7c36ca46-e60a-43de-8715-48bb06e40cdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:21:41.839377 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c36ca46-e60a-43de-8715-48bb06e40cdb None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:21:41.844237 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c594b9-dbc7-47b1-acec-b8801c493bc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:21:41.845731 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c594b9-dbc7-47b1-acec-b8801c493bc2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:41.846024 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c594b9-dbc7-47b1-acec-b8801c493bc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:41.846382 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c594b9-dbc7-47b1-acec-b8801c493bc2 admin admin] PATH_INFO: `/v3/projects/d9103a1dee084ce49690cbb3f72ad65b` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:41.847485 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c594b9-dbc7-47b1-acec-b8801c493bc2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d9103a1dee084ce49690cbb3f72ad65b)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:21:41.849552 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c594b9-dbc7-47b1-acec-b8801c493bc2 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:21:41.858476 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:21:41.906034 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-e3c594b9-dbc7-47b1-acec-b8801c493bc2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd9103a1dee084ce49690cbb3f72ad65b', 'request_id': 'req-e3c594b9-dbc7-47b1-acec-b8801c493bc2'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:41.913682 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-e3c594b9-dbc7-47b1-acec-b8801c493bc2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd9103a1dee084ce49690cbb3f72ad65b', 'request_id': 'req-e3c594b9-dbc7-47b1-acec-b8801c493bc2'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:41.919344 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-e3c594b9-dbc7-47b1-acec-b8801c493bc2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd9103a1dee084ce49690cbb3f72ad65b', 'request_id': 'req-e3c594b9-dbc7-47b1-acec-b8801c493bc2'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:41.930429 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-e3c594b9-dbc7-47b1-acec-b8801c493bc2 admin admin] The token cache is being invalidate because project d9103a1dee084ce49690cbb3f72ad65b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 12:21:41.930725 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-e3c594b9-dbc7-47b1-acec-b8801c493bc2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:41.931365 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-e3c594b9-dbc7-47b1-acec-b8801c493bc2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:41.932946 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1311/2620] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Jul 25 12:21:41 2022] DELETE /identity/v3/projects/d9103a1dee084ce49690cbb3f72ad65b => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:21:41.939354 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ff5d96-4ea9-4795-b92a-8be2f5e71c9b None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:41.939681 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ff5d96-4ea9-4795-b92a-8be2f5e71c9b None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:41.939965 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ff5d96-4ea9-4795-b92a-8be2f5e71c9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:41.958935 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-73ff5d96-4ea9-4795-b92a-8be2f5e71c9b None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:21:41.960786 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-73ff5d96-4ea9-4795-b92a-8be2f5e71c9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:21:42.003431 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:21:42.021241 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1310/2621] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:21:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:21:42.029952 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b945e3b-3dbf-4389-bfc7-20d1cfc23983 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:42.030246 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b945e3b-3dbf-4389-bfc7-20d1cfc23983 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:42.030536 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b945e3b-3dbf-4389-bfc7-20d1cfc23983 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:42.049759 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-4b945e3b-3dbf-4389-bfc7-20d1cfc23983 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:21:42.050960 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-4b945e3b-3dbf-4389-bfc7-20d1cfc23983 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:21:42.059284 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1312/2622] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:21:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:21:42.063417 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ff5d96-4ea9-4795-b92a-8be2f5e71c9b None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:21:42.073649 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ff5d96-4ea9-4795-b92a-8be2f5e71c9b None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:21:42.078184 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec08ae2-2b18-446b-9c49-a796d075f7b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:21:42.078833 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec08ae2-2b18-446b-9c49-a796d075f7b0 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:42.079123 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec08ae2-2b18-446b-9c49-a796d075f7b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:42.079407 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec08ae2-2b18-446b-9c49-a796d075f7b0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:42.080169 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec08ae2-2b18-446b-9c49-a796d075f7b0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:21:42.080482 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec08ae2-2b18-446b-9c49-a796d075f7b0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:21:42.082068 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec08ae2-2b18-446b-9c49-a796d075f7b0 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:21:42.094429 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1311/2623] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Jul 25 12:21:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:21:42.098532 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b945e3b-3dbf-4389-bfc7-20d1cfc23983 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:21:42.108272 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b945e3b-3dbf-4389-bfc7-20d1cfc23983 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:21:42.113453 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f75838a-9849-4dc3-b426-0ee99e551856 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:21:42.114164 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f75838a-9849-4dc3-b426-0ee99e551856 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:42.114453 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f75838a-9849-4dc3-b426-0ee99e551856 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:42.114736 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f75838a-9849-4dc3-b426-0ee99e551856 admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:42.115426 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f75838a-9849-4dc3-b426-0ee99e551856 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:21:42.117364 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f75838a-9849-4dc3-b426-0ee99e551856 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:21:42.144036 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1313/2624] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Jul 25 12:21:42 2022] POST /identity/v3/projects => generated 379 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:21:42.148972 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec08ae2-2b18-446b-9c49-a796d075f7b0 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:21:42.159530 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec08ae2-2b18-446b-9c49-a796d075f7b0 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:21:42.216169 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3401b5a2-649a-4390-a624-efb82f3b194d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:21:42.216844 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3401b5a2-649a-4390-a624-efb82f3b194d admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:42.217133 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3401b5a2-649a-4390-a624-efb82f3b194d admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:42.217416 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3401b5a2-649a-4390-a624-efb82f3b194d admin admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:42.218122 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3401b5a2-649a-4390-a624-efb82f3b194d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:21:42.220077 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3401b5a2-649a-4390-a624-efb82f3b194d admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:21:42.240477 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1312/2625] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Jul 25 12:21:42 2022] POST /identity/v3/users => generated 290 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:21:42.244575 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f75838a-9849-4dc3-b426-0ee99e551856 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:21:42.254821 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f75838a-9849-4dc3-b426-0ee99e551856 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:21:42.260197 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a3913c-ff07-4ccc-8677-99ca2d5181d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:21:42.260891 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a3913c-ff07-4ccc-8677-99ca2d5181d9 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:42.261269 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a3913c-ff07-4ccc-8677-99ca2d5181d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:42.261442 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a3913c-ff07-4ccc-8677-99ca2d5181d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:42.262167 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a3913c-ff07-4ccc-8677-99ca2d5181d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:21:42.263808 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a3913c-ff07-4ccc-8677-99ca2d5181d9 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:21:42.274657 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1314/2626] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Jul 25 12:21:42 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:21:42.278448 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3401b5a2-649a-4390-a624-efb82f3b194d admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:21:42.288232 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3401b5a2-649a-4390-a624-efb82f3b194d admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:21:42.292909 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eda6c86-e195-483a-8278-7bdbfa549b70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:21:42.293560 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eda6c86-e195-483a-8278-7bdbfa549b70 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:42.293849 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eda6c86-e195-483a-8278-7bdbfa549b70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:42.294134 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eda6c86-e195-483a-8278-7bdbfa549b70 admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:42.294858 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eda6c86-e195-483a-8278-7bdbfa549b70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:21:42.296487 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eda6c86-e195-483a-8278-7bdbfa549b70 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:21:42.306805 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1313/2627] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Jul 25 12:21:42 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:21:42.310689 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a3913c-ff07-4ccc-8677-99ca2d5181d9 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:21:42.322348 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a3913c-ff07-4ccc-8677-99ca2d5181d9 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:21:42.325745 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1629be9d-0892-44e9-b76e-cdd43b0b32e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:21:42.326633 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1629be9d-0892-44e9-b76e-cdd43b0b32e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:42.326998 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1629be9d-0892-44e9-b76e-cdd43b0b32e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:42.327281 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1629be9d-0892-44e9-b76e-cdd43b0b32e8 admin admin] PATH_INFO: `/v3/projects/141003bbc94b41569683aef4c15b46f1/users/3494ce75df62497bb737254fd1665a3b/roles/472313ae69174578824df0bed6390744` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:42.343299 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1629be9d-0892-44e9-b76e-cdd43b0b32e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=141003bbc94b41569683aef4c15b46f1, user_id=3494ce75df62497bb737254fd1665a3b, role_id=472313ae69174578824df0bed6390744)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:21:42.346001 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1629be9d-0892-44e9-b76e-cdd43b0b32e8 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:21:42.358451 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1315/2628] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Jul 25 12:21:42 2022] PUT /identity/v3/projects/141003bbc94b41569683aef4c15b46f1/users/3494ce75df62497bb737254fd1665a3b/roles/472313ae69174578824df0bed6390744 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:21:42.363407 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1879e2f4-2fb2-4b78-9a4b-d3d744b25681 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:42.363739 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1879e2f4-2fb2-4b78-9a4b-d3d744b25681 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:42.364032 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1879e2f4-2fb2-4b78-9a4b-d3d744b25681 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:42.384160 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-1879e2f4-2fb2-4b78-9a4b-d3d744b25681 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:21:42.385395 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-1879e2f4-2fb2-4b78-9a4b-d3d744b25681 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:21:42.448418 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1314/2629] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:21:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:21:42.458851 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1629be9d-0892-44e9-b76e-cdd43b0b32e8 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:21:42.462695 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-1629be9d-0892-44e9-b76e-cdd43b0b32e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:21:42.527374 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1629be9d-0892-44e9-b76e-cdd43b0b32e8 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:21:42.531087 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9ddbda-cfda-4365-8476-4e651aa5c0ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51733781201243e8abf9eca39b207750', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:21:42.531768 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9ddbda-cfda-4365-8476-4e651aa5c0ab service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:42.532054 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9ddbda-cfda-4365-8476-4e651aa5c0ab service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:42.532338 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9ddbda-cfda-4365-8476-4e651aa5c0ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:42.541763 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9ddbda-cfda-4365-8476-4e651aa5c0ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:21:42.543262 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9ddbda-cfda-4365-8476-4e651aa5c0ab service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:21:42.554469 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1316/2630] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:21:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 12:21:42.800812 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1879e2f4-2fb2-4b78-9a4b-d3d744b25681 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:21:42.802499 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-1879e2f4-2fb2-4b78-9a4b-d3d744b25681 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:21:42.869762 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1879e2f4-2fb2-4b78-9a4b-d3d744b25681 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:21:42.873633 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fcc410-4dc4-4137-8232-7f095ae06fdf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f89ce6f0cf04373a414d927fa0f8a7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:21:42.874281 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fcc410-4dc4-4137-8232-7f095ae06fdf service cinder] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:42.874569 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fcc410-4dc4-4137-8232-7f095ae06fdf service cinder] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:42.874845 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fcc410-4dc4-4137-8232-7f095ae06fdf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:42.876686 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-a4fcc410-4dc4-4137-8232-7f095ae06fdf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:21:42.886090 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:21:42.951459 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4fcc410-4dc4-4137-8232-7f095ae06fdf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:21:42.953583 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4fcc410-4dc4-4137-8232-7f095ae06fdf service cinder] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:21:42.963815 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1315/2631] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:21:42 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:21:43.541700 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9ddbda-cfda-4365-8476-4e651aa5c0ab service neutron] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:21:43.551988 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9ddbda-cfda-4365-8476-4e651aa5c0ab service neutron] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:21:43.556345 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e150bba-10f1-450a-9949-8c4e98d6d6f1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f89ce6f0cf04373a414d927fa0f8a7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:21:43.557118 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e150bba-10f1-450a-9949-8c4e98d6d6f1 service cinder] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:43.557424 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e150bba-10f1-450a-9949-8c4e98d6d6f1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:43.557704 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e150bba-10f1-450a-9949-8c4e98d6d6f1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:43.559885 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-2e150bba-10f1-450a-9949-8c4e98d6d6f1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:21:43.598637 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:21:43.638116 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e150bba-10f1-450a-9949-8c4e98d6d6f1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:21:43.639707 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e150bba-10f1-450a-9949-8c4e98d6d6f1 service cinder] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:21:43.651643 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1317/2632] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:21:43 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:21:47.353128 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc918a1-56d8-4a28-854e-94dc3d5dda2b None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:47.354291 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc918a1-56d8-4a28-854e-94dc3d5dda2b None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:47.354830 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc918a1-56d8-4a28-854e-94dc3d5dda2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:47.407075 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-8cc918a1-56d8-4a28-854e-94dc3d5dda2b None None] MFA Rules not processed for user `3494ce75df62497bb737254fd1665a3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:21:47.408465 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-8cc918a1-56d8-4a28-854e-94dc3d5dda2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:21:47.548349 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1316/2633] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:21:47 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:21:47.548680 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e150bba-10f1-450a-9949-8c4e98d6d6f1 service cinder] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:21:47.587836 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e150bba-10f1-450a-9949-8c4e98d6d6f1 service cinder] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:21:47.603464 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389a989a-f623-4116-9fdb-bff39566dfda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:21:47.604198 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389a989a-f623-4116-9fdb-bff39566dfda admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:47.604494 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389a989a-f623-4116-9fdb-bff39566dfda admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:47.604780 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389a989a-f623-4116-9fdb-bff39566dfda admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:47.605486 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389a989a-f623-4116-9fdb-bff39566dfda admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:21:47.607493 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389a989a-f623-4116-9fdb-bff39566dfda admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:21:47.650880 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1318/2634] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Jul 25 12:21:47 2022] POST /identity/v3/projects => generated 377 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:21:47.655665 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc918a1-56d8-4a28-854e-94dc3d5dda2b None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:21:47.670833 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc918a1-56d8-4a28-854e-94dc3d5dda2b None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:21:47.742164 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f01adaa-8600-4ab8-9263-8100e88dff4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:21:47.742880 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f01adaa-8600-4ab8-9263-8100e88dff4c admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:47.743237 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f01adaa-8600-4ab8-9263-8100e88dff4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:47.743635 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f01adaa-8600-4ab8-9263-8100e88dff4c admin admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:47.744583 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f01adaa-8600-4ab8-9263-8100e88dff4c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:21:47.746655 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f01adaa-8600-4ab8-9263-8100e88dff4c admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:21:47.792475 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389a989a-f623-4116-9fdb-bff39566dfda admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:21:47.793623 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1317/2635] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Jul 25 12:21:47 2022] POST /identity/v3/users => generated 295 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 12:21:47.831706 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f01adaa-8600-4ab8-9263-8100e88dff4c admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:21:47.835216 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-5f01adaa-8600-4ab8-9263-8100e88dff4c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:21:47.846120 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389a989a-f623-4116-9fdb-bff39566dfda admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:21:47.867448 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f01adaa-8600-4ab8-9263-8100e88dff4c admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:21:47.873597 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a079778f-3df7-48ef-a3ed-1466d557631e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:21:47.874465 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3a6f73-0dbb-4bc5-a3c2-9e4d9d16a441 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:21:47.874916 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a079778f-3df7-48ef-a3ed-1466d557631e admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:47.875395 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a079778f-3df7-48ef-a3ed-1466d557631e admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:47.875892 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a079778f-3df7-48ef-a3ed-1466d557631e admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:47.876096 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3a6f73-0dbb-4bc5-a3c2-9e4d9d16a441 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:47.876586 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3a6f73-0dbb-4bc5-a3c2-9e4d9d16a441 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:47.877019 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3a6f73-0dbb-4bc5-a3c2-9e4d9d16a441 admin admin] PATH_INFO: `/v3/users/504aef7b26144da3953fad61d759894b` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:47.878553 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3a6f73-0dbb-4bc5-a3c2-9e4d9d16a441 admin admin] RBAC: Authorizing `identity:delete_user(user_id=504aef7b26144da3953fad61d759894b)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:21:47.882612 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3a6f73-0dbb-4bc5-a3c2-9e4d9d16a441 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:21:47.884385 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a079778f-3df7-48ef-a3ed-1466d557631e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:21:47.886310 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a079778f-3df7-48ef-a3ed-1466d557631e admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:21:47.912778 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1319/2636] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Jul 25 12:21:47 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:21:47.924638 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a079778f-3df7-48ef-a3ed-1466d557631e admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:21:47.941137 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=93,keystone:INSERT=3,keystone:DELETE=4 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:21:47.952218 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a079778f-3df7-48ef-a3ed-1466d557631e admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:21:47.962238 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050a1def-6d56-4e2d-a294-417ff78d5dda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:21:47.964264 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050a1def-6d56-4e2d-a294-417ff78d5dda admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:47.965106 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050a1def-6d56-4e2d-a294-417ff78d5dda admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:47.965649 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050a1def-6d56-4e2d-a294-417ff78d5dda admin admin] PATH_INFO: `/v3/projects/5922b8528101418b84ca9b5b53f49cc5/users/28e5eae4e13945768c0c440cfbf7d429/roles/a7d20885b6d94928a34b3cc5b3a00006` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:47.990885 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050a1def-6d56-4e2d-a294-417ff78d5dda admin admin] RBAC: Authorizing `identity:create_grant(project_id=5922b8528101418b84ca9b5b53f49cc5, user_id=28e5eae4e13945768c0c440cfbf7d429, role_id=a7d20885b6d94928a34b3cc5b3a00006)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:21:47.994341 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050a1def-6d56-4e2d-a294-417ff78d5dda admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:21:48.006793 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-5e3a6f73-0dbb-4bc5-a3c2-9e4d9d16a441 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '504aef7b26144da3953fad61d759894b'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:48.007721 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-5e3a6f73-0dbb-4bc5-a3c2-9e4d9d16a441 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '504aef7b26144da3953fad61d759894b'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:48.024162 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1320/2637] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Jul 25 12:21:47 2022] PUT /identity/v3/projects/5922b8528101418b84ca9b5b53f49cc5/users/28e5eae4e13945768c0c440cfbf7d429/roles/a7d20885b6d94928a34b3cc5b3a00006 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:21:48.026536 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-5e3a6f73-0dbb-4bc5-a3c2-9e4d9d16a441 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '504aef7b26144da3953fad61d759894b'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:48.034989 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050a1def-6d56-4e2d-a294-417ff78d5dda admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:21:48.052392 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-5e3a6f73-0dbb-4bc5-a3c2-9e4d9d16a441 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '504aef7b26144da3953fad61d759894b'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:48.073980 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-5e3a6f73-0dbb-4bc5-a3c2-9e4d9d16a441 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '504aef7b26144da3953fad61d759894b'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:48.091423 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1318/2638] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Jul 25 12:21:47 2022] DELETE /identity/v3/users/504aef7b26144da3953fad61d759894b => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:21:48.098752 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66b471a-82df-4d8b-a25d-50fa8834362c None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:48.101591 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66b471a-82df-4d8b-a25d-50fa8834362c None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:48.101759 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66b471a-82df-4d8b-a25d-50fa8834362c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:48.135661 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050a1def-6d56-4e2d-a294-417ff78d5dda admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:21:48.138971 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-f66b471a-82df-4d8b-a25d-50fa8834362c None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:21:48.141847 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-f66b471a-82df-4d8b-a25d-50fa8834362c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:21:48.208110 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547c4e31-9acb-4e6a-9381-70451e04077a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:21:48.209277 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547c4e31-9acb-4e6a-9381-70451e04077a admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:48.209277 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547c4e31-9acb-4e6a-9381-70451e04077a admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:48.209620 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547c4e31-9acb-4e6a-9381-70451e04077a admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:48.210440 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547c4e31-9acb-4e6a-9381-70451e04077a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:21:48.212689 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547c4e31-9acb-4e6a-9381-70451e04077a admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:21:48.226423 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1321/2639] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Jul 25 12:21:48 2022] GET /identity/v3/roles => generated 1420 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:21:48.232421 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547c4e31-9acb-4e6a-9381-70451e04077a admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:21:48.236871 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-547c4e31-9acb-4e6a-9381-70451e04077a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:21:48.258052 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1319/2640] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:21:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:21:48.264112 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547c4e31-9acb-4e6a-9381-70451e04077a admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:21:48.271943 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9be79ed-2059-47a3-9ab0-c3114f87ad5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:21:48.275151 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66b471a-82df-4d8b-a25d-50fa8834362c None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:21:48.277163 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-f66b471a-82df-4d8b-a25d-50fa8834362c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:21:48.277472 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9be79ed-2059-47a3-9ab0-c3114f87ad5f admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:48.277774 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9be79ed-2059-47a3-9ab0-c3114f87ad5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:48.278215 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9be79ed-2059-47a3-9ab0-c3114f87ad5f admin admin] PATH_INFO: `/v3/projects/5922b8528101418b84ca9b5b53f49cc5/users/28e5eae4e13945768c0c440cfbf7d429/roles/a7d20885b6d94928a34b3cc5b3a00006` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:48.285080 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9be79ed-2059-47a3-9ab0-c3114f87ad5f admin admin] RBAC: Authorizing `identity:create_grant(project_id=5922b8528101418b84ca9b5b53f49cc5, user_id=28e5eae4e13945768c0c440cfbf7d429, role_id=a7d20885b6d94928a34b3cc5b3a00006)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:21:48.288759 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9be79ed-2059-47a3-9ab0-c3114f87ad5f admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:21:48.301013 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1322/2641] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Jul 25 12:21:48 2022] PUT /identity/v3/projects/5922b8528101418b84ca9b5b53f49cc5/users/28e5eae4e13945768c0c440cfbf7d429/roles/a7d20885b6d94928a34b3cc5b3a00006 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:21:48.388935 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66b471a-82df-4d8b-a25d-50fa8834362c None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:21:48.448891 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de60c97-1aaa-47bd-8ad5-ead5f039694e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51733781201243e8abf9eca39b207750', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:21:48.450207 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de60c97-1aaa-47bd-8ad5-ead5f039694e service neutron] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:48.450699 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de60c97-1aaa-47bd-8ad5-ead5f039694e service neutron] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:48.451774 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de60c97-1aaa-47bd-8ad5-ead5f039694e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:48.476739 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:21:48.587775 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de60c97-1aaa-47bd-8ad5-ead5f039694e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:21:48.589590 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de60c97-1aaa-47bd-8ad5-ead5f039694e service neutron] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:21:48.617711 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1320/2642] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:21:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:21:49.165987 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9be79ed-2059-47a3-9ab0-c3114f87ad5f admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:21:49.168183 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-a9be79ed-2059-47a3-9ab0-c3114f87ad5f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:21:49.179846 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:21:49.204713 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9be79ed-2059-47a3-9ab0-c3114f87ad5f admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:21:49.216947 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e35c3d-ec95-4881-831f-fa1d95e1b828 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:21:49.218039 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e35c3d-ec95-4881-831f-fa1d95e1b828 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:49.218601 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e35c3d-ec95-4881-831f-fa1d95e1b828 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:49.218798 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e35c3d-ec95-4881-831f-fa1d95e1b828 admin admin] PATH_INFO: `/v3/projects/d8529c14d4164488b57600795f3ac919` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:49.227267 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e35c3d-ec95-4881-831f-fa1d95e1b828 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d8529c14d4164488b57600795f3ac919)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:21:49.233734 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e35c3d-ec95-4881-831f-fa1d95e1b828 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:21:49.343952 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-03e35c3d-ec95-4881-831f-fa1d95e1b828 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd8529c14d4164488b57600795f3ac919', 'request_id': 'req-03e35c3d-ec95-4881-831f-fa1d95e1b828'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:49.366211 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-03e35c3d-ec95-4881-831f-fa1d95e1b828 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd8529c14d4164488b57600795f3ac919', 'request_id': 'req-03e35c3d-ec95-4881-831f-fa1d95e1b828'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:49.388210 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-03e35c3d-ec95-4881-831f-fa1d95e1b828 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd8529c14d4164488b57600795f3ac919', 'request_id': 'req-03e35c3d-ec95-4881-831f-fa1d95e1b828'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:49.422186 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-03e35c3d-ec95-4881-831f-fa1d95e1b828 admin admin] The token cache is being invalidate because project d8529c14d4164488b57600795f3ac919 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 12:21:49.422481 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-03e35c3d-ec95-4881-831f-fa1d95e1b828 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:49.424069 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-03e35c3d-ec95-4881-831f-fa1d95e1b828 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:49.427193 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1323/2643] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Jul 25 12:21:49 2022] DELETE /identity/v3/projects/d8529c14d4164488b57600795f3ac919 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:21:49.433083 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de60c97-1aaa-47bd-8ad5-ead5f039694e service neutron] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:21:49.469638 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de60c97-1aaa-47bd-8ad5-ead5f039694e service neutron] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:21:49.581076 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fae17ee-87f3-4358-a2f5-992c5c2b60bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:21:49.583670 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fae17ee-87f3-4358-a2f5-992c5c2b60bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:49.584206 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fae17ee-87f3-4358-a2f5-992c5c2b60bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:49.584871 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fae17ee-87f3-4358-a2f5-992c5c2b60bf admin admin] PATH_INFO: `/v3/users/abf3ff102f6b4f9182aeaa4767ff1022` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:49.590303 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fae17ee-87f3-4358-a2f5-992c5c2b60bf admin admin] RBAC: Authorizing `identity:delete_user(user_id=abf3ff102f6b4f9182aeaa4767ff1022)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:21:49.600695 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fae17ee-87f3-4358-a2f5-992c5c2b60bf admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:21:49.708043 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-0fae17ee-87f3-4358-a2f5-992c5c2b60bf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'abf3ff102f6b4f9182aeaa4767ff1022'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:49.709310 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-0fae17ee-87f3-4358-a2f5-992c5c2b60bf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'abf3ff102f6b4f9182aeaa4767ff1022'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:49.721874 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-0fae17ee-87f3-4358-a2f5-992c5c2b60bf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'abf3ff102f6b4f9182aeaa4767ff1022'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:49.744396 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-0fae17ee-87f3-4358-a2f5-992c5c2b60bf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'abf3ff102f6b4f9182aeaa4767ff1022'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:49.753165 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-0fae17ee-87f3-4358-a2f5-992c5c2b60bf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'abf3ff102f6b4f9182aeaa4767ff1022'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:49.761978 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1321/2644] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Jul 25 12:21:49 2022] DELETE /identity/v3/users/abf3ff102f6b4f9182aeaa4767ff1022 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:21:50.561113 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e35c3d-ec95-4881-831f-fa1d95e1b828 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:21:50.562840 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-03e35c3d-ec95-4881-831f-fa1d95e1b828 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:21:50.588559 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e35c3d-ec95-4881-831f-fa1d95e1b828 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:21:50.704712 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f300893-a68e-41eb-b9be-4a893ba1cceb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:21:50.705781 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f300893-a68e-41eb-b9be-4a893ba1cceb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:50.706074 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f300893-a68e-41eb-b9be-4a893ba1cceb admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:50.706364 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f300893-a68e-41eb-b9be-4a893ba1cceb admin admin] PATH_INFO: `/v3/projects/43328f65d95a45f383b61da6af7b4639` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:50.708188 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f300893-a68e-41eb-b9be-4a893ba1cceb admin admin] RBAC: Authorizing `identity:delete_project(project_id=43328f65d95a45f383b61da6af7b4639)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:21:50.710950 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f300893-a68e-41eb-b9be-4a893ba1cceb admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:21:50.805936 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-9f300893-a68e-41eb-b9be-4a893ba1cceb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '43328f65d95a45f383b61da6af7b4639', 'request_id': 'req-9f300893-a68e-41eb-b9be-4a893ba1cceb'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:50.816941 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-9f300893-a68e-41eb-b9be-4a893ba1cceb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '43328f65d95a45f383b61da6af7b4639', 'request_id': 'req-9f300893-a68e-41eb-b9be-4a893ba1cceb'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:50.839027 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-9f300893-a68e-41eb-b9be-4a893ba1cceb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '43328f65d95a45f383b61da6af7b4639', 'request_id': 'req-9f300893-a68e-41eb-b9be-4a893ba1cceb'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:50.853508 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-9f300893-a68e-41eb-b9be-4a893ba1cceb admin admin] The token cache is being invalidate because project 43328f65d95a45f383b61da6af7b4639 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 12:21:50.853819 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-9f300893-a68e-41eb-b9be-4a893ba1cceb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:50.854602 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-9f300893-a68e-41eb-b9be-4a893ba1cceb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:50.859076 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1324/2645] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Jul 25 12:21:50 2022] DELETE /identity/v3/projects/43328f65d95a45f383b61da6af7b4639 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:21:50.860416 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fae17ee-87f3-4358-a2f5-992c5c2b60bf admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:21:50.862350 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-0fae17ee-87f3-4358-a2f5-992c5c2b60bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:21:50.880449 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fae17ee-87f3-4358-a2f5-992c5c2b60bf admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:21:50.917099 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:21:51.002349 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d23ac9-43fb-49b2-99db-9a51e02af1b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:21:51.005083 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d23ac9-43fb-49b2-99db-9a51e02af1b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:51.006328 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d23ac9-43fb-49b2-99db-9a51e02af1b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:51.006823 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d23ac9-43fb-49b2-99db-9a51e02af1b4 admin admin] PATH_INFO: `/v3/users/c8bc7cfbb6674e24860e8f07fd2da44a` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:51.010339 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d23ac9-43fb-49b2-99db-9a51e02af1b4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c8bc7cfbb6674e24860e8f07fd2da44a)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:21:51.013757 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d23ac9-43fb-49b2-99db-9a51e02af1b4 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:21:51.103518 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-d6d23ac9-43fb-49b2-99db-9a51e02af1b4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c8bc7cfbb6674e24860e8f07fd2da44a'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:51.105783 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-d6d23ac9-43fb-49b2-99db-9a51e02af1b4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c8bc7cfbb6674e24860e8f07fd2da44a'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:51.122057 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-d6d23ac9-43fb-49b2-99db-9a51e02af1b4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c8bc7cfbb6674e24860e8f07fd2da44a'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:51.167971 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-d6d23ac9-43fb-49b2-99db-9a51e02af1b4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c8bc7cfbb6674e24860e8f07fd2da44a'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:51.190380 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-d6d23ac9-43fb-49b2-99db-9a51e02af1b4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c8bc7cfbb6674e24860e8f07fd2da44a'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:51.202874 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1322/2646] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Jul 25 12:21:50 2022] DELETE /identity/v3/users/c8bc7cfbb6674e24860e8f07fd2da44a => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:21:51.733659 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f300893-a68e-41eb-b9be-4a893ba1cceb admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:21:51.734939 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-9f300893-a68e-41eb-b9be-4a893ba1cceb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:21:51.748644 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f300893-a68e-41eb-b9be-4a893ba1cceb admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:21:51.797471 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:21:51.872642 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d23ac9-43fb-49b2-99db-9a51e02af1b4 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:21:51.874931 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-d6d23ac9-43fb-49b2-99db-9a51e02af1b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:21:51.890995 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050b5c02-7490-4050-885a-783ba76f5525 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:21:51.892105 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050b5c02-7490-4050-885a-783ba76f5525 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:51.892473 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050b5c02-7490-4050-885a-783ba76f5525 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:51.892840 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050b5c02-7490-4050-885a-783ba76f5525 admin admin] PATH_INFO: `/v3/users/4ef93840ca1e499ab87431dcba856308` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:51.895903 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050b5c02-7490-4050-885a-783ba76f5525 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4ef93840ca1e499ab87431dcba856308)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:21:51.900268 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050b5c02-7490-4050-885a-783ba76f5525 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:21:51.910493 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d23ac9-43fb-49b2-99db-9a51e02af1b4 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:21:51.926291 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566c8e60-e477-48c5-9171-c5be2f684634 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:21:51.927835 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566c8e60-e477-48c5-9171-c5be2f684634 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:51.929540 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566c8e60-e477-48c5-9171-c5be2f684634 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:51.930285 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566c8e60-e477-48c5-9171-c5be2f684634 admin admin] PATH_INFO: `/v3/projects/461a8b9163fc4912bf72fab49ea7fc22` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:51.937018 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566c8e60-e477-48c5-9171-c5be2f684634 admin admin] RBAC: Authorizing `identity:delete_project(project_id=461a8b9163fc4912bf72fab49ea7fc22)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:21:51.941554 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566c8e60-e477-48c5-9171-c5be2f684634 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:21:52.068397 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-050b5c02-7490-4050-885a-783ba76f5525 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4ef93840ca1e499ab87431dcba856308'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:52.071007 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-566c8e60-e477-48c5-9171-c5be2f684634 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '461a8b9163fc4912bf72fab49ea7fc22', 'request_id': 'req-566c8e60-e477-48c5-9171-c5be2f684634'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:52.088307 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-050b5c02-7490-4050-885a-783ba76f5525 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4ef93840ca1e499ab87431dcba856308'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:52.096443 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-566c8e60-e477-48c5-9171-c5be2f684634 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '461a8b9163fc4912bf72fab49ea7fc22', 'request_id': 'req-566c8e60-e477-48c5-9171-c5be2f684634'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:52.109279 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-050b5c02-7490-4050-885a-783ba76f5525 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4ef93840ca1e499ab87431dcba856308'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:52.113757 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-566c8e60-e477-48c5-9171-c5be2f684634 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '461a8b9163fc4912bf72fab49ea7fc22', 'request_id': 'req-566c8e60-e477-48c5-9171-c5be2f684634'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:52.146736 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-050b5c02-7490-4050-885a-783ba76f5525 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4ef93840ca1e499ab87431dcba856308'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:52.148547 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-050b5c02-7490-4050-885a-783ba76f5525 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4ef93840ca1e499ab87431dcba856308'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:52.160465 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-566c8e60-e477-48c5-9171-c5be2f684634 admin admin] The token cache is being invalidate because project 461a8b9163fc4912bf72fab49ea7fc22 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80298) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 12:21:52.163480 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-566c8e60-e477-48c5-9171-c5be2f684634 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:52.164762 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-566c8e60-e477-48c5-9171-c5be2f684634 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:52.167441 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1325/2647] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Jul 25 12:21:51 2022] DELETE /identity/v3/users/4ef93840ca1e499ab87431dcba856308 => generated 0 bytes in 431 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:21:52.168566 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1323/2648] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Jul 25 12:21:51 2022] DELETE /identity/v3/projects/461a8b9163fc4912bf72fab49ea7fc22 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:21:52.173999 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47455a1c-b4de-4883-a176-a1a2a9dd5e0e None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:52.174163 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47455a1c-b4de-4883-a176-a1a2a9dd5e0e None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:52.174374 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47455a1c-b4de-4883-a176-a1a2a9dd5e0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:52.214134 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-47455a1c-b4de-4883-a176-a1a2a9dd5e0e None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:21:52.215471 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-47455a1c-b4de-4883-a176-a1a2a9dd5e0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:21:52.378691 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1326/2649] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:21:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:21:52.380071 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc9c923-4ffb-4cdd-b910-007dcf1f208a None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:52.381624 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc9c923-4ffb-4cdd-b910-007dcf1f208a None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:52.382290 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc9c923-4ffb-4cdd-b910-007dcf1f208a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:52.426657 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-7cc9c923-4ffb-4cdd-b910-007dcf1f208a None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:21:52.428959 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-7cc9c923-4ffb-4cdd-b910-007dcf1f208a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:21:52.452841 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1324/2650] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:21:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:21:52.453364 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47455a1c-b4de-4883-a176-a1a2a9dd5e0e None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:21:52.471649 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47455a1c-b4de-4883-a176-a1a2a9dd5e0e None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:21:52.494422 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8f20c2-1431-4b93-859d-cb0be318b02c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:21:52.497512 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8f20c2-1431-4b93-859d-cb0be318b02c admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:52.498566 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8f20c2-1431-4b93-859d-cb0be318b02c admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:52.499874 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8f20c2-1431-4b93-859d-cb0be318b02c admin admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:52.501996 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8f20c2-1431-4b93-859d-cb0be318b02c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:21:52.502609 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8f20c2-1431-4b93-859d-cb0be318b02c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:21:52.504862 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8f20c2-1431-4b93-859d-cb0be318b02c admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:21:52.532101 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1327/2651] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Jul 25 12:21:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:21:52.540030 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc9c923-4ffb-4cdd-b910-007dcf1f208a None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:21:52.544767 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8f20c2-1431-4b93-859d-cb0be318b02c admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:21:52.546891 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-2f8f20c2-1431-4b93-859d-cb0be318b02c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:21:52.551413 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc9c923-4ffb-4cdd-b910-007dcf1f208a None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:21:52.558031 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e562c7-5002-4620-b8ef-aed11fad5af8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:21:52.559091 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e562c7-5002-4620-b8ef-aed11fad5af8 admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:52.559668 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e562c7-5002-4620-b8ef-aed11fad5af8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:52.560197 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e562c7-5002-4620-b8ef-aed11fad5af8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:52.561119 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e562c7-5002-4620-b8ef-aed11fad5af8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:21:52.563804 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8f20c2-1431-4b93-859d-cb0be318b02c admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:21:52.567047 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e562c7-5002-4620-b8ef-aed11fad5af8 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:21:52.571335 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095eae56-e75b-451f-a4ac-6e174efef6de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:21:52.572256 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095eae56-e75b-451f-a4ac-6e174efef6de admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:52.575293 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095eae56-e75b-451f-a4ac-6e174efef6de admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:52.575620 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095eae56-e75b-451f-a4ac-6e174efef6de admin admin] PATH_INFO: `/v3/projects/e9eaa14479b64763a86720ccf40e395a` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:52.576973 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095eae56-e75b-451f-a4ac-6e174efef6de admin admin] RBAC: Authorizing `identity:delete_project(project_id=e9eaa14479b64763a86720ccf40e395a)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:21:52.579294 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095eae56-e75b-451f-a4ac-6e174efef6de admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:21:52.617998 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:21:52.624203 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=80,keystone:DELETE=17,keystone:INSERT=3 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:21:52.639839 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1325/2652] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Jul 25 12:21:52 2022] POST /identity/v3/projects => generated 373 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:21:52.645994 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e562c7-5002-4620-b8ef-aed11fad5af8 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:21:52.694238 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-095eae56-e75b-451f-a4ac-6e174efef6de admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e9eaa14479b64763a86720ccf40e395a', 'request_id': 'req-095eae56-e75b-451f-a4ac-6e174efef6de'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:52.710277 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-095eae56-e75b-451f-a4ac-6e174efef6de admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e9eaa14479b64763a86720ccf40e395a', 'request_id': 'req-095eae56-e75b-451f-a4ac-6e174efef6de'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:52.720270 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-095eae56-e75b-451f-a4ac-6e174efef6de admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e9eaa14479b64763a86720ccf40e395a', 'request_id': 'req-095eae56-e75b-451f-a4ac-6e174efef6de'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:52.757445 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-095eae56-e75b-451f-a4ac-6e174efef6de admin admin] The token cache is being invalidate because project e9eaa14479b64763a86720ccf40e395a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 12:21:52.758077 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-095eae56-e75b-451f-a4ac-6e174efef6de admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:52.762056 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-095eae56-e75b-451f-a4ac-6e174efef6de admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:52.768036 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1328/2653] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Jul 25 12:21:52 2022] DELETE /identity/v3/projects/e9eaa14479b64763a86720ccf40e395a => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:21:52.769391 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095eae56-e75b-451f-a4ac-6e174efef6de admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:21:52.774340 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-095eae56-e75b-451f-a4ac-6e174efef6de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:21:52.798116 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e562c7-5002-4620-b8ef-aed11fad5af8 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:21:52.891131 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5714195-fec3-4954-b671-09ff18a4e0c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:21:52.891131 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5714195-fec3-4954-b671-09ff18a4e0c1 admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:52.891897 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5714195-fec3-4954-b671-09ff18a4e0c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:52.891897 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5714195-fec3-4954-b671-09ff18a4e0c1 admin admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:52.892388 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5714195-fec3-4954-b671-09ff18a4e0c1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:21:52.894913 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5714195-fec3-4954-b671-09ff18a4e0c1 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:21:52.930551 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095eae56-e75b-451f-a4ac-6e174efef6de admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:21:52.936499 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1326/2654] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Jul 25 12:21:52 2022] POST /identity/v3/users => generated 287 bytes in 290 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:21:52.937428 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5714195-fec3-4954-b671-09ff18a4e0c1 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:21:52.943584 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-e5714195-fec3-4954-b671-09ff18a4e0c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:21:52.948709 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad340fa-47dc-401c-b1c0-26b0e26beb01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:21:52.949615 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad340fa-47dc-401c-b1c0-26b0e26beb01 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:52.949911 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad340fa-47dc-401c-b1c0-26b0e26beb01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:52.950193 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad340fa-47dc-401c-b1c0-26b0e26beb01 admin admin] PATH_INFO: `/v3/users/c41698cb7564488595cc9d70a23312b7` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:52.954147 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad340fa-47dc-401c-b1c0-26b0e26beb01 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c41698cb7564488595cc9d70a23312b7)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:21:52.961996 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad340fa-47dc-401c-b1c0-26b0e26beb01 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:21:52.984797 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5714195-fec3-4954-b671-09ff18a4e0c1 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:21:53.006337 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a345ad71-5cdb-4f44-8186-f7ac405fa0e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:21:53.007282 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a345ad71-5cdb-4f44-8186-f7ac405fa0e2 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:53.007599 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a345ad71-5cdb-4f44-8186-f7ac405fa0e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:53.007901 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a345ad71-5cdb-4f44-8186-f7ac405fa0e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:53.008632 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a345ad71-5cdb-4f44-8186-f7ac405fa0e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:21:53.010571 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a345ad71-5cdb-4f44-8186-f7ac405fa0e2 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:21:53.035750 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1327/2655] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Jul 25 12:21:52 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:21:53.038236 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a345ad71-5cdb-4f44-8186-f7ac405fa0e2 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:21:53.064747 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a345ad71-5cdb-4f44-8186-f7ac405fa0e2 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:21:53.075569 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b5646f-51b7-430f-b90a-90bacc779a5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:21:53.076723 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b5646f-51b7-430f-b90a-90bacc779a5a admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:53.077372 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b5646f-51b7-430f-b90a-90bacc779a5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:53.078938 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b5646f-51b7-430f-b90a-90bacc779a5a admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:53.079644 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-dad340fa-47dc-401c-b1c0-26b0e26beb01 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c41698cb7564488595cc9d70a23312b7'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:53.081915 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b5646f-51b7-430f-b90a-90bacc779a5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:21:53.086439 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b5646f-51b7-430f-b90a-90bacc779a5a admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:21:53.100591 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-dad340fa-47dc-401c-b1c0-26b0e26beb01 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c41698cb7564488595cc9d70a23312b7'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:53.112355 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1328/2656] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Jul 25 12:21:53 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:21:53.113324 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b5646f-51b7-430f-b90a-90bacc779a5a admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:21:53.119398 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-dad340fa-47dc-401c-b1c0-26b0e26beb01 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c41698cb7564488595cc9d70a23312b7'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:53.135964 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-dad340fa-47dc-401c-b1c0-26b0e26beb01 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c41698cb7564488595cc9d70a23312b7'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:53.136170 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b5646f-51b7-430f-b90a-90bacc779a5a admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:21:53.136727 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-dad340fa-47dc-401c-b1c0-26b0e26beb01 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c41698cb7564488595cc9d70a23312b7'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:53.141649 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee7d721-385f-4c36-83ff-0947491b0230 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:21:53.142631 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee7d721-385f-4c36-83ff-0947491b0230 admin admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:53.142970 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee7d721-385f-4c36-83ff-0947491b0230 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:53.143258 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee7d721-385f-4c36-83ff-0947491b0230 admin admin] PATH_INFO: `/v3/projects/f6b1d49d16c741d195afa089f5f1eb36/users/922702e951594912bea9546b33ed0335/roles/472313ae69174578824df0bed6390744` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:53.145139 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1329/2657] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Jul 25 12:21:52 2022] DELETE /identity/v3/users/c41698cb7564488595cc9d70a23312b7 => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:21:53.189680 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee7d721-385f-4c36-83ff-0947491b0230 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f6b1d49d16c741d195afa089f5f1eb36, user_id=922702e951594912bea9546b33ed0335, role_id=472313ae69174578824df0bed6390744)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:21:53.191899 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee7d721-385f-4c36-83ff-0947491b0230 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:21:53.229279 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1329/2658] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Jul 25 12:21:53 2022] PUT /identity/v3/projects/f6b1d49d16c741d195afa089f5f1eb36/users/922702e951594912bea9546b33ed0335/roles/472313ae69174578824df0bed6390744 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:21:53.243005 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53802107-a5b6-4d34-9612-d461166556c9 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:53.243299 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53802107-a5b6-4d34-9612-d461166556c9 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:53.243599 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53802107-a5b6-4d34-9612-d461166556c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:53.300437 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-53802107-a5b6-4d34-9612-d461166556c9 None None] MFA Rules not processed for user `922702e951594912bea9546b33ed0335`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:21:53.301802 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-53802107-a5b6-4d34-9612-d461166556c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:21:53.330241 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:21:53.453156 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1330/2659] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:21:53 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:21:53.467744 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee7d721-385f-4c36-83ff-0947491b0230 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:21:53.470165 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-5ee7d721-385f-4c36-83ff-0947491b0230 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:21:53.505792 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53802107-a5b6-4d34-9612-d461166556c9 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:21:53.507274 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-53802107-a5b6-4d34-9612-d461166556c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:21:53.610798 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53802107-a5b6-4d34-9612-d461166556c9 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:21:53.615943 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e41e44-203a-4447-a357-8da5c178316e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51733781201243e8abf9eca39b207750', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:21:53.616621 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e41e44-203a-4447-a357-8da5c178316e service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:53.617516 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e41e44-203a-4447-a357-8da5c178316e service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:53.617815 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e41e44-203a-4447-a357-8da5c178316e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:53.620068 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-f2e41e44-203a-4447-a357-8da5c178316e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:21:53.639097 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee7d721-385f-4c36-83ff-0947491b0230 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:21:53.653802 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a07307-dc72-418f-8d9e-4da7c91bdd54 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f89ce6f0cf04373a414d927fa0f8a7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:21:53.655154 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a07307-dc72-418f-8d9e-4da7c91bdd54 service cinder] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:53.655463 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a07307-dc72-418f-8d9e-4da7c91bdd54 service cinder] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:53.655905 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a07307-dc72-418f-8d9e-4da7c91bdd54 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:53.670442 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:21:53.678065 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a07307-dc72-418f-8d9e-4da7c91bdd54 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:21:53.691146 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a07307-dc72-418f-8d9e-4da7c91bdd54 service cinder] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:21:53.711710 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e41e44-203a-4447-a357-8da5c178316e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:21:53.723471 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1330/2660] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:21:53 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 256 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:21:53.725187 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a07307-dc72-418f-8d9e-4da7c91bdd54 service cinder] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:21:53.726502 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-63a07307-dc72-418f-8d9e-4da7c91bdd54 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:21:53.730638 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e41e44-203a-4447-a357-8da5c178316e service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:21:53.754878 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a07307-dc72-418f-8d9e-4da7c91bdd54 service cinder] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:21:53.759627 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1331/2661] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:21:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 253 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:21:53.772205 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339e70ed-d36c-4247-9f5a-78661ce4508d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:21:53.772205 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339e70ed-d36c-4247-9f5a-78661ce4508d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:53.772205 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339e70ed-d36c-4247-9f5a-78661ce4508d admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:53.772205 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339e70ed-d36c-4247-9f5a-78661ce4508d admin admin] PATH_INFO: `/v3/projects/cc5f40813e434b6ebbf6908896b992f0` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:53.774726 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339e70ed-d36c-4247-9f5a-78661ce4508d admin admin] RBAC: Authorizing `identity:delete_project(project_id=cc5f40813e434b6ebbf6908896b992f0)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:21:53.780856 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339e70ed-d36c-4247-9f5a-78661ce4508d admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:21:53.869160 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-339e70ed-d36c-4247-9f5a-78661ce4508d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cc5f40813e434b6ebbf6908896b992f0', 'request_id': 'req-339e70ed-d36c-4247-9f5a-78661ce4508d'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:53.889366 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-339e70ed-d36c-4247-9f5a-78661ce4508d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cc5f40813e434b6ebbf6908896b992f0', 'request_id': 'req-339e70ed-d36c-4247-9f5a-78661ce4508d'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:53.901705 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-339e70ed-d36c-4247-9f5a-78661ce4508d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cc5f40813e434b6ebbf6908896b992f0', 'request_id': 'req-339e70ed-d36c-4247-9f5a-78661ce4508d'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:53.928164 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-339e70ed-d36c-4247-9f5a-78661ce4508d admin admin] The token cache is being invalidate because project cc5f40813e434b6ebbf6908896b992f0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80298) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 12:21:53.928752 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-339e70ed-d36c-4247-9f5a-78661ce4508d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:53.930629 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-339e70ed-d36c-4247-9f5a-78661ce4508d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:53.938894 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1331/2662] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Jul 25 12:21:53 2022] DELETE /identity/v3/projects/cc5f40813e434b6ebbf6908896b992f0 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:21:53.940788 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42e02ec-b139-4aff-bcfe-3734867be90a None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:53.941236 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42e02ec-b139-4aff-bcfe-3734867be90a None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:53.941535 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42e02ec-b139-4aff-bcfe-3734867be90a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:53.970399 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-e42e02ec-b139-4aff-bcfe-3734867be90a None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:21:53.972013 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-e42e02ec-b139-4aff-bcfe-3734867be90a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:21:54.154226 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1332/2663] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:21:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:21:54.167416 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc128a8-8b4f-45d2-a1ad-8808de6cbbf3 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:54.167748 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc128a8-8b4f-45d2-a1ad-8808de6cbbf3 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:54.170308 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc128a8-8b4f-45d2-a1ad-8808de6cbbf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:54.238667 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-2fc128a8-8b4f-45d2-a1ad-8808de6cbbf3 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:21:54.249596 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-2fc128a8-8b4f-45d2-a1ad-8808de6cbbf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:21:54.295032 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1332/2664] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:21:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:21:54.295233 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42e02ec-b139-4aff-bcfe-3734867be90a None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:21:54.301615 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:21:54.310482 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42e02ec-b139-4aff-bcfe-3734867be90a None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:21:54.334163 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0b20ed-9581-403f-acb3-e865e90a32c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:21:54.335098 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0b20ed-9581-403f-acb3-e865e90a32c3 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:54.335473 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0b20ed-9581-403f-acb3-e865e90a32c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:54.335886 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0b20ed-9581-403f-acb3-e865e90a32c3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:54.336699 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0b20ed-9581-403f-acb3-e865e90a32c3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:21:54.337082 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0b20ed-9581-403f-acb3-e865e90a32c3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:21:54.338899 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0b20ed-9581-403f-acb3-e865e90a32c3 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:21:54.376002 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1333/2665] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Jul 25 12:21:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:21:54.380170 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc128a8-8b4f-45d2-a1ad-8808de6cbbf3 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:21:54.391538 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc128a8-8b4f-45d2-a1ad-8808de6cbbf3 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:21:54.404804 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e922b7-b0eb-49e6-aaf7-9ecc5e639ba6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:21:54.405082 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e922b7-b0eb-49e6-aaf7-9ecc5e639ba6 admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:54.405376 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e922b7-b0eb-49e6-aaf7-9ecc5e639ba6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:54.405662 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e922b7-b0eb-49e6-aaf7-9ecc5e639ba6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:54.406376 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e922b7-b0eb-49e6-aaf7-9ecc5e639ba6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:21:54.408326 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e922b7-b0eb-49e6-aaf7-9ecc5e639ba6 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:21:54.474539 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1333/2666] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Jul 25 12:21:54 2022] POST /identity/v3/projects => generated 371 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:21:54.479001 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0b20ed-9581-403f-acb3-e865e90a32c3 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:21:54.498788 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0b20ed-9581-403f-acb3-e865e90a32c3 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:21:54.589154 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7ee0b3-e6f6-4ee8-bf36-96779b8c5533 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:21:54.589780 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7ee0b3-e6f6-4ee8-bf36-96779b8c5533 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:54.590172 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7ee0b3-e6f6-4ee8-bf36-96779b8c5533 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:54.590557 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7ee0b3-e6f6-4ee8-bf36-96779b8c5533 admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:54.591381 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7ee0b3-e6f6-4ee8-bf36-96779b8c5533 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:21:54.593577 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7ee0b3-e6f6-4ee8-bf36-96779b8c5533 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:21:54.635725 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1334/2667] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Jul 25 12:21:54 2022] POST /identity/v3/users => generated 286 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 12:21:54.637103 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e922b7-b0eb-49e6-aaf7-9ecc5e639ba6 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:21:54.653265 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e922b7-b0eb-49e6-aaf7-9ecc5e639ba6 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:21:54.668499 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc1ce1d-1432-424d-bd09-c2dab43d3acb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:21:54.670216 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc1ce1d-1432-424d-bd09-c2dab43d3acb admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:54.670642 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc1ce1d-1432-424d-bd09-c2dab43d3acb admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:54.671479 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc1ce1d-1432-424d-bd09-c2dab43d3acb admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:54.673092 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc1ce1d-1432-424d-bd09-c2dab43d3acb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:21:54.676693 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc1ce1d-1432-424d-bd09-c2dab43d3acb admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:21:54.695421 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1334/2668] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Jul 25 12:21:54 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:21:54.701389 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7ee0b3-e6f6-4ee8-bf36-96779b8c5533 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:21:54.715365 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7ee0b3-e6f6-4ee8-bf36-96779b8c5533 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:21:54.720963 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52fed23-4a46-43fb-9185-4ab009026a93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:21:54.721869 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52fed23-4a46-43fb-9185-4ab009026a93 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:54.722205 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52fed23-4a46-43fb-9185-4ab009026a93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:54.722587 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52fed23-4a46-43fb-9185-4ab009026a93 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:54.723465 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52fed23-4a46-43fb-9185-4ab009026a93 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:21:54.725996 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52fed23-4a46-43fb-9185-4ab009026a93 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:21:54.742353 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1335/2669] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Jul 25 12:21:54 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:21:54.743320 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc1ce1d-1432-424d-bd09-c2dab43d3acb admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:21:54.756539 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc1ce1d-1432-424d-bd09-c2dab43d3acb admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:21:54.780599 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036ccf12-54fe-412c-8ec5-eafdfc50d530 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:21:54.782684 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036ccf12-54fe-412c-8ec5-eafdfc50d530 admin admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:54.784399 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036ccf12-54fe-412c-8ec5-eafdfc50d530 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:54.785033 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036ccf12-54fe-412c-8ec5-eafdfc50d530 admin admin] PATH_INFO: `/v3/projects/c5e2dcaea7f743e285390b2c43f6157e/users/a57c1f2c57704cbdb3733e7a75a7ccee/roles/472313ae69174578824df0bed6390744` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:54.809808 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036ccf12-54fe-412c-8ec5-eafdfc50d530 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c5e2dcaea7f743e285390b2c43f6157e, user_id=a57c1f2c57704cbdb3733e7a75a7ccee, role_id=472313ae69174578824df0bed6390744)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:21:54.813391 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036ccf12-54fe-412c-8ec5-eafdfc50d530 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:21:54.845610 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1335/2670] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Jul 25 12:21:54 2022] PUT /identity/v3/projects/c5e2dcaea7f743e285390b2c43f6157e/users/a57c1f2c57704cbdb3733e7a75a7ccee/roles/472313ae69174578824df0bed6390744 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:21:54.913914 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70039784-bdd2-4e95-99dc-ac396cb665b0 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:54.914472 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70039784-bdd2-4e95-99dc-ac396cb665b0 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:54.914472 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70039784-bdd2-4e95-99dc-ac396cb665b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:54.948127 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-70039784-bdd2-4e95-99dc-ac396cb665b0 None None] MFA Rules not processed for user `a57c1f2c57704cbdb3733e7a75a7ccee`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:21:54.949369 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-70039784-bdd2-4e95-99dc-ac396cb665b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:21:55.040116 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1336/2671] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:21:54 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:21:55.056377 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036ccf12-54fe-412c-8ec5-eafdfc50d530 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:21:55.060097 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-036ccf12-54fe-412c-8ec5-eafdfc50d530 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:21:55.183034 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036ccf12-54fe-412c-8ec5-eafdfc50d530 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:21:55.190295 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8043ee5f-c0b6-442f-8dce-c03fa09de72b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f89ce6f0cf04373a414d927fa0f8a7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:21:55.191259 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8043ee5f-c0b6-442f-8dce-c03fa09de72b service cinder] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:55.191595 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8043ee5f-c0b6-442f-8dce-c03fa09de72b service cinder] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:55.192039 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8043ee5f-c0b6-442f-8dce-c03fa09de72b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:55.210437 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8043ee5f-c0b6-442f-8dce-c03fa09de72b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:21:55.212926 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8043ee5f-c0b6-442f-8dce-c03fa09de72b service cinder] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:21:55.252123 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1336/2672] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:21:55 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 12:21:55.252581 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70039784-bdd2-4e95-99dc-ac396cb665b0 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:21:55.424175 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928676c5-beee-4dfd-8ae4-48de5859d51a None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:55.424671 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928676c5-beee-4dfd-8ae4-48de5859d51a None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:55.425127 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928676c5-beee-4dfd-8ae4-48de5859d51a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:55.435686 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70039784-bdd2-4e95-99dc-ac396cb665b0 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:21:55.458624 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60eea1d1-5706-46af-8474-b63f24396608 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:21:55.459722 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60eea1d1-5706-46af-8474-b63f24396608 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:55.460113 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60eea1d1-5706-46af-8474-b63f24396608 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:55.460495 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60eea1d1-5706-46af-8474-b63f24396608 admin admin] PATH_INFO: `/v3/users/a57c1f2c57704cbdb3733e7a75a7ccee` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:55.468475 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60eea1d1-5706-46af-8474-b63f24396608 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a57c1f2c57704cbdb3733e7a75a7ccee)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:21:55.470604 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60eea1d1-5706-46af-8474-b63f24396608 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:21:55.478010 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:21:55.501305 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-928676c5-beee-4dfd-8ae4-48de5859d51a None None] MFA Rules not processed for user `28e5eae4e13945768c0c440cfbf7d429`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:21:55.504211 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-928676c5-beee-4dfd-8ae4-48de5859d51a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:21:55.512536 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:21:55.575923 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-60eea1d1-5706-46af-8474-b63f24396608 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a57c1f2c57704cbdb3733e7a75a7ccee'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:55.593980 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-60eea1d1-5706-46af-8474-b63f24396608 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a57c1f2c57704cbdb3733e7a75a7ccee'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:55.610065 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-60eea1d1-5706-46af-8474-b63f24396608 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a57c1f2c57704cbdb3733e7a75a7ccee'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:55.634887 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-60eea1d1-5706-46af-8474-b63f24396608 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a57c1f2c57704cbdb3733e7a75a7ccee'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:55.636240 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-60eea1d1-5706-46af-8474-b63f24396608 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a57c1f2c57704cbdb3733e7a75a7ccee'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:55.651723 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1337/2673] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Jul 25 12:21:55 2022] DELETE /identity/v3/users/a57c1f2c57704cbdb3733e7a75a7ccee => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 12:21:55.653310 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fc39ea-8634-41ad-a705-1598f3662882 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:55.653830 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fc39ea-8634-41ad-a705-1598f3662882 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:55.654156 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fc39ea-8634-41ad-a705-1598f3662882 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:55.675398 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1337/2674] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:21:55 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:21:55.677348 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef714c4-3961-4840-ac15-900ff6f115dd None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:55.677651 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef714c4-3961-4840-ac15-900ff6f115dd None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:55.677941 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef714c4-3961-4840-ac15-900ff6f115dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:55.681388 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-60fc39ea-8634-41ad-a705-1598f3662882 None None] MFA Rules not processed for user `922702e951594912bea9546b33ed0335`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:21:55.683499 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-60fc39ea-8634-41ad-a705-1598f3662882 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:21:55.720609 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-7ef714c4-3961-4840-ac15-900ff6f115dd None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:21:55.721778 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-7ef714c4-3961-4840-ac15-900ff6f115dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:21:55.761393 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1338/2675] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:21:55 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:21:55.762640 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fc39ea-8634-41ad-a705-1598f3662882 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:21:55.764301 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-60fc39ea-8634-41ad-a705-1598f3662882 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:21:55.867002 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1338/2676] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:21:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:21:55.882046 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef714c4-3961-4840-ac15-900ff6f115dd None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:21:55.883649 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-7ef714c4-3961-4840-ac15-900ff6f115dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:21:55.899062 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fc39ea-8634-41ad-a705-1598f3662882 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:21:55.910438 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc6caeb-d929-4943-9bcd-7ac3c4bc2173 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f67142a1193d47c7bd33d7040268233c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:21:55.912015 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc6caeb-d929-4943-9bcd-7ac3c4bc2173 service nova] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:55.912513 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc6caeb-d929-4943-9bcd-7ac3c4bc2173 service nova] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:55.912944 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc6caeb-d929-4943-9bcd-7ac3c4bc2173 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:55.915453 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-7cc6caeb-d929-4943-9bcd-7ac3c4bc2173 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:21:55.958700 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:21:56.027892 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef714c4-3961-4840-ac15-900ff6f115dd None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:21:56.037940 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad0f1ba-eb16-42d9-8f8e-e84d4b10de05 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51733781201243e8abf9eca39b207750', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:21:56.038863 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad0f1ba-eb16-42d9-8f8e-e84d4b10de05 service neutron] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:56.039156 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad0f1ba-eb16-42d9-8f8e-e84d4b10de05 service neutron] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:56.039631 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad0f1ba-eb16-42d9-8f8e-e84d4b10de05 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:56.053606 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad0f1ba-eb16-42d9-8f8e-e84d4b10de05 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:21:56.054462 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc6caeb-d929-4943-9bcd-7ac3c4bc2173 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:21:56.054983 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad0f1ba-eb16-42d9-8f8e-e84d4b10de05 service neutron] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:21:56.056779 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc6caeb-d929-4943-9bcd-7ac3c4bc2173 service nova] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:21:56.068176 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1339/2677] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:21:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 12:21:56.075101 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1339/2678] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:21:55 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:21:56.644712 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad0f1ba-eb16-42d9-8f8e-e84d4b10de05 service neutron] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:21:56.647342 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-7ad0f1ba-eb16-42d9-8f8e-e84d4b10de05 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:21:56.690829 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad0f1ba-eb16-42d9-8f8e-e84d4b10de05 service neutron] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:21:56.698697 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b2da39-e196-473a-ad97-a0ae81f8f787 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:21:56.699910 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b2da39-e196-473a-ad97-a0ae81f8f787 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:56.700421 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b2da39-e196-473a-ad97-a0ae81f8f787 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:56.700998 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b2da39-e196-473a-ad97-a0ae81f8f787 admin admin] PATH_INFO: `/v3/projects/c5e2dcaea7f743e285390b2c43f6157e` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:56.702491 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b2da39-e196-473a-ad97-a0ae81f8f787 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c5e2dcaea7f743e285390b2c43f6157e)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:21:56.704782 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b2da39-e196-473a-ad97-a0ae81f8f787 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:21:56.760657 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:21:56.784166 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-30b2da39-e196-473a-ad97-a0ae81f8f787 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c5e2dcaea7f743e285390b2c43f6157e', 'request_id': 'req-30b2da39-e196-473a-ad97-a0ae81f8f787'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:56.805201 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-30b2da39-e196-473a-ad97-a0ae81f8f787 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c5e2dcaea7f743e285390b2c43f6157e', 'request_id': 'req-30b2da39-e196-473a-ad97-a0ae81f8f787'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:56.813860 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-30b2da39-e196-473a-ad97-a0ae81f8f787 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c5e2dcaea7f743e285390b2c43f6157e', 'request_id': 'req-30b2da39-e196-473a-ad97-a0ae81f8f787'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:56.837657 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-30b2da39-e196-473a-ad97-a0ae81f8f787 admin admin] The token cache is being invalidate because project c5e2dcaea7f743e285390b2c43f6157e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80298) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 12:21:56.838091 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-30b2da39-e196-473a-ad97-a0ae81f8f787 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:56.847966 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-30b2da39-e196-473a-ad97-a0ae81f8f787 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:56.851127 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1340/2679] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Jul 25 12:21:56 2022] DELETE /identity/v3/projects/c5e2dcaea7f743e285390b2c43f6157e => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:21:56.856939 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043d7f22-8b5b-4bf0-980c-843dbe5a6e3d None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:56.857521 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043d7f22-8b5b-4bf0-980c-843dbe5a6e3d None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:56.858046 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043d7f22-8b5b-4bf0-980c-843dbe5a6e3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:56.893234 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-043d7f22-8b5b-4bf0-980c-843dbe5a6e3d None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:21:56.894952 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-043d7f22-8b5b-4bf0-980c-843dbe5a6e3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:21:57.030439 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1340/2680] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:21:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:21:57.031701 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a12edc6-c795-42d1-b1aa-98dfa3cfd892 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:57.032003 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a12edc6-c795-42d1-b1aa-98dfa3cfd892 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:57.032293 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a12edc6-c795-42d1-b1aa-98dfa3cfd892 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:57.069250 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-8a12edc6-c795-42d1-b1aa-98dfa3cfd892 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:21:57.070483 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-8a12edc6-c795-42d1-b1aa-98dfa3cfd892 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:21:57.087264 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1341/2681] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:21:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:21:57.092229 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043d7f22-8b5b-4bf0-980c-843dbe5a6e3d None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:21:57.107988 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043d7f22-8b5b-4bf0-980c-843dbe5a6e3d None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:21:57.116416 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928e5d59-7e03-4007-87e4-931425b54086 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:21:57.117468 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928e5d59-7e03-4007-87e4-931425b54086 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:57.117890 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928e5d59-7e03-4007-87e4-931425b54086 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:57.124825 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928e5d59-7e03-4007-87e4-931425b54086 admin admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:57.126153 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928e5d59-7e03-4007-87e4-931425b54086 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:21:57.127256 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928e5d59-7e03-4007-87e4-931425b54086 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:21:57.128948 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928e5d59-7e03-4007-87e4-931425b54086 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:21:57.158079 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1341/2682] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Jul 25 12:21:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:21:57.159372 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a12edc6-c795-42d1-b1aa-98dfa3cfd892 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:21:57.171896 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a12edc6-c795-42d1-b1aa-98dfa3cfd892 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:21:57.182118 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53998187-85e2-4cc0-807a-9aee58eb092b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:21:57.183182 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53998187-85e2-4cc0-807a-9aee58eb092b admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:57.183809 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53998187-85e2-4cc0-807a-9aee58eb092b admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:57.184298 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53998187-85e2-4cc0-807a-9aee58eb092b admin admin] PATH_INFO: `/v3/projects` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:57.185293 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53998187-85e2-4cc0-807a-9aee58eb092b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:21:57.187777 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53998187-85e2-4cc0-807a-9aee58eb092b admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:21:57.253853 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928e5d59-7e03-4007-87e4-931425b54086 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:21:57.254556 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1342/2683] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Jul 25 12:21:57 2022] POST /identity/v3/projects => generated 375 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 12:21:57.284248 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928e5d59-7e03-4007-87e4-931425b54086 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:21:57.428149 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee1503b-b60c-4c7b-8462-3e78d49339a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:21:57.428931 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee1503b-b60c-4c7b-8462-3e78d49339a7 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:57.429234 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee1503b-b60c-4c7b-8462-3e78d49339a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:57.429525 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee1503b-b60c-4c7b-8462-3e78d49339a7 admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:57.430353 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee1503b-b60c-4c7b-8462-3e78d49339a7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:21:57.433786 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee1503b-b60c-4c7b-8462-3e78d49339a7 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:21:57.469182 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1342/2684] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Jul 25 12:21:57 2022] POST /identity/v3/users => generated 288 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:21:57.473413 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53998187-85e2-4cc0-807a-9aee58eb092b admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:21:57.494241 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53998187-85e2-4cc0-807a-9aee58eb092b admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:21:57.515402 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8de1ee2-979d-4517-8d00-db29676bd476 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:21:57.516252 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8de1ee2-979d-4517-8d00-db29676bd476 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:57.516800 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8de1ee2-979d-4517-8d00-db29676bd476 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:57.517277 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8de1ee2-979d-4517-8d00-db29676bd476 admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:57.518201 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8de1ee2-979d-4517-8d00-db29676bd476 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:21:57.520466 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8de1ee2-979d-4517-8d00-db29676bd476 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:21:57.549898 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1343/2685] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Jul 25 12:21:57 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:21:57.554522 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee1503b-b60c-4c7b-8462-3e78d49339a7 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:21:57.569215 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee1503b-b60c-4c7b-8462-3e78d49339a7 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:21:57.575027 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffe3b2b-53db-43a1-8714-6e8cee629b01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:21:57.575470 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffe3b2b-53db-43a1-8714-6e8cee629b01 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:57.575820 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffe3b2b-53db-43a1-8714-6e8cee629b01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:57.576096 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffe3b2b-53db-43a1-8714-6e8cee629b01 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:57.576867 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ffe3b2b-53db-43a1-8714-6e8cee629b01 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:21:57.578602 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ffe3b2b-53db-43a1-8714-6e8cee629b01 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:21:57.592856 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1343/2686] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Jul 25 12:21:57 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:21:57.596761 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8de1ee2-979d-4517-8d00-db29676bd476 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:21:57.612823 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8de1ee2-979d-4517-8d00-db29676bd476 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:21:57.630872 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c89740-6b43-453e-9624-7239e69aaa31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:21:57.631851 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c89740-6b43-453e-9624-7239e69aaa31 admin admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:57.632345 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c89740-6b43-453e-9624-7239e69aaa31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:57.632448 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c89740-6b43-453e-9624-7239e69aaa31 admin admin] PATH_INFO: `/v3/projects/d7bc9b29547841d8b46f264148eae000/users/a170b4b5faed40f188ce6751904f109a/roles/472313ae69174578824df0bed6390744` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:57.670030 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c89740-6b43-453e-9624-7239e69aaa31 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d7bc9b29547841d8b46f264148eae000, user_id=a170b4b5faed40f188ce6751904f109a, role_id=472313ae69174578824df0bed6390744)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:21:57.673060 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c89740-6b43-453e-9624-7239e69aaa31 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:21:57.700497 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1344/2687] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Jul 25 12:21:57 2022] PUT /identity/v3/projects/d7bc9b29547841d8b46f264148eae000/users/a170b4b5faed40f188ce6751904f109a/roles/472313ae69174578824df0bed6390744 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:21:57.711930 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cb4efd-a1a4-41c3-9c94-72d77bf82e8a None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:57.712218 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cb4efd-a1a4-41c3-9c94-72d77bf82e8a None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:57.712496 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cb4efd-a1a4-41c3-9c94-72d77bf82e8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:57.780276 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-38cb4efd-a1a4-41c3-9c94-72d77bf82e8a None None] MFA Rules not processed for user `a170b4b5faed40f188ce6751904f109a`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:21:57.783977 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-38cb4efd-a1a4-41c3-9c94-72d77bf82e8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:21:57.801712 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:21:57.918443 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1344/2688] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:21:57 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:21:57.930377 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c89740-6b43-453e-9624-7239e69aaa31 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:21:57.936463 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-a3c89740-6b43-453e-9624-7239e69aaa31 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:21:58.064682 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c89740-6b43-453e-9624-7239e69aaa31 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:21:58.072146 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b88714d-9e5d-44a4-a55a-4b5226a48325 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ea16528b1374bae9cf54603f7907bfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:21:58.073859 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b88714d-9e5d-44a4-a55a-4b5226a48325 service glance] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:58.074389 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b88714d-9e5d-44a4-a55a-4b5226a48325 service glance] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:58.074739 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b88714d-9e5d-44a4-a55a-4b5226a48325 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:58.087876 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b88714d-9e5d-44a4-a55a-4b5226a48325 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:21:58.089709 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b88714d-9e5d-44a4-a55a-4b5226a48325 service glance] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:21:58.100624 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1345/2689] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:21:57 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:21:59.634341 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cb4efd-a1a4-41c3-9c94-72d77bf82e8a None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:21:59.644390 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-38cb4efd-a1a4-41c3-9c94-72d77bf82e8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:21:59.827142 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cb4efd-a1a4-41c3-9c94-72d77bf82e8a None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:21:59.834113 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b28a388-f6ef-4173-9a72-ff52f07c8ec4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:21:59.840330 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b28a388-f6ef-4173-9a72-ff52f07c8ec4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:21:59.840330 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b28a388-f6ef-4173-9a72-ff52f07c8ec4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:21:59.840330 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b28a388-f6ef-4173-9a72-ff52f07c8ec4 admin admin] PATH_INFO: `/v3/users/d2d6f10a6fe841e5b1d12bce78f97d91` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:21:59.840330 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b28a388-f6ef-4173-9a72-ff52f07c8ec4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d2d6f10a6fe841e5b1d12bce78f97d91)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:21:59.840330 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b28a388-f6ef-4173-9a72-ff52f07c8ec4 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:21:59.903351 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-4b28a388-f6ef-4173-9a72-ff52f07c8ec4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd2d6f10a6fe841e5b1d12bce78f97d91'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:59.912031 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-4b28a388-f6ef-4173-9a72-ff52f07c8ec4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd2d6f10a6fe841e5b1d12bce78f97d91'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:59.924215 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-4b28a388-f6ef-4173-9a72-ff52f07c8ec4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd2d6f10a6fe841e5b1d12bce78f97d91'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:59.947080 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-4b28a388-f6ef-4173-9a72-ff52f07c8ec4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd2d6f10a6fe841e5b1d12bce78f97d91'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:59.948709 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-4b28a388-f6ef-4173-9a72-ff52f07c8ec4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd2d6f10a6fe841e5b1d12bce78f97d91'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:21:59.965192 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1345/2690] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Jul 25 12:21:59 2022] DELETE /identity/v3/users/d2d6f10a6fe841e5b1d12bce78f97d91 => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:22:00.211918 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b88714d-9e5d-44a4-a55a-4b5226a48325 service glance] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:22:00.226122 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b88714d-9e5d-44a4-a55a-4b5226a48325 service glance] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:22:00.293132 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8720dde2-3efe-46d4-b562-a5228661b37a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:22:00.293998 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8720dde2-3efe-46d4-b562-a5228661b37a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:22:00.294282 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8720dde2-3efe-46d4-b562-a5228661b37a admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:22:00.294560 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8720dde2-3efe-46d4-b562-a5228661b37a admin admin] PATH_INFO: `/v3/projects/aef3a5bd8ffa43718ea02647f1f74fc8` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:22:00.295944 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8720dde2-3efe-46d4-b562-a5228661b37a admin admin] RBAC: Authorizing `identity:delete_project(project_id=aef3a5bd8ffa43718ea02647f1f74fc8)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:22:00.297849 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8720dde2-3efe-46d4-b562-a5228661b37a admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:22:00.306820 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:22:00.392162 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-8720dde2-3efe-46d4-b562-a5228661b37a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aef3a5bd8ffa43718ea02647f1f74fc8', 'request_id': 'req-8720dde2-3efe-46d4-b562-a5228661b37a'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:22:00.410065 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-8720dde2-3efe-46d4-b562-a5228661b37a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aef3a5bd8ffa43718ea02647f1f74fc8', 'request_id': 'req-8720dde2-3efe-46d4-b562-a5228661b37a'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:22:00.418742 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-8720dde2-3efe-46d4-b562-a5228661b37a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aef3a5bd8ffa43718ea02647f1f74fc8', 'request_id': 'req-8720dde2-3efe-46d4-b562-a5228661b37a'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:22:00.437928 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-8720dde2-3efe-46d4-b562-a5228661b37a admin admin] The token cache is being invalidate because project aef3a5bd8ffa43718ea02647f1f74fc8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80298) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 12:22:00.438471 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-8720dde2-3efe-46d4-b562-a5228661b37a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:22:00.439930 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-8720dde2-3efe-46d4-b562-a5228661b37a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:22:00.441785 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1346/2691] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Jul 25 12:22:00 2022] DELETE /identity/v3/projects/aef3a5bd8ffa43718ea02647f1f74fc8 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:22:00.446947 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1d27ba-99a0-4b3f-aa4a-e3cda9c202b0 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:22:00.447465 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1d27ba-99a0-4b3f-aa4a-e3cda9c202b0 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:22:00.447931 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1d27ba-99a0-4b3f-aa4a-e3cda9c202b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:22:00.473451 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-ba1d27ba-99a0-4b3f-aa4a-e3cda9c202b0 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:22:00.474709 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-ba1d27ba-99a0-4b3f-aa4a-e3cda9c202b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:22:00.514249 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:22:00.569765 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1346/2692] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:22:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 12:22:00.578866 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b24a608-d559-4ffd-9f46-747ed6a9a60c None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:22:00.579170 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b24a608-d559-4ffd-9f46-747ed6a9a60c None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:22:00.579462 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b24a608-d559-4ffd-9f46-747ed6a9a60c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:22:00.627344 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-8b24a608-d559-4ffd-9f46-747ed6a9a60c None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:22:00.628832 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-8b24a608-d559-4ffd-9f46-747ed6a9a60c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:22:00.642537 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1347/2693] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:22:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:22:00.643846 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1d27ba-99a0-4b3f-aa4a-e3cda9c202b0 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:22:00.661720 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1d27ba-99a0-4b3f-aa4a-e3cda9c202b0 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:22:00.668857 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1493f8-f449-4bd8-bf13-806640e10c73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:22:00.669554 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1493f8-f449-4bd8-bf13-806640e10c73 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:22:00.669842 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1493f8-f449-4bd8-bf13-806640e10c73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:22:00.670124 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1493f8-f449-4bd8-bf13-806640e10c73 admin admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:22:00.671019 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1493f8-f449-4bd8-bf13-806640e10c73 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:22:00.671334 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1493f8-f449-4bd8-bf13-806640e10c73 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:22:00.673318 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1493f8-f449-4bd8-bf13-806640e10c73 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:22:00.694030 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1347/2694] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Jul 25 12:22:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:22:00.695620 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b24a608-d559-4ffd-9f46-747ed6a9a60c None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:22:00.706612 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b24a608-d559-4ffd-9f46-747ed6a9a60c None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:22:00.713673 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9128fa-7218-455d-9889-c8606d5cc0f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:22:00.714687 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9128fa-7218-455d-9889-c8606d5cc0f0 admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:22:00.715509 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9128fa-7218-455d-9889-c8606d5cc0f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:22:00.716084 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9128fa-7218-455d-9889-c8606d5cc0f0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:22:00.717172 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9128fa-7218-455d-9889-c8606d5cc0f0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:22:00.720951 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9128fa-7218-455d-9889-c8606d5cc0f0 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:22:00.795327 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1493f8-f449-4bd8-bf13-806640e10c73 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:22:00.817226 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1348/2695] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Jul 25 12:22:00 2022] POST /identity/v3/projects => generated 371 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 12:22:00.826878 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1493f8-f449-4bd8-bf13-806640e10c73 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:22:00.913285 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd29371-03e6-4b5d-bda8-8ccb52415e1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:22:00.914084 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd29371-03e6-4b5d-bda8-8ccb52415e1e admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:22:00.914974 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd29371-03e6-4b5d-bda8-8ccb52415e1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:22:00.914974 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd29371-03e6-4b5d-bda8-8ccb52415e1e admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:22:00.915735 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd29371-03e6-4b5d-bda8-8ccb52415e1e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:22:00.918490 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd29371-03e6-4b5d-bda8-8ccb52415e1e admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:22:00.953043 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1348/2696] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Jul 25 12:22:00 2022] POST /identity/v3/users => generated 286 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:22:00.957794 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9128fa-7218-455d-9889-c8606d5cc0f0 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:22:00.971762 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9128fa-7218-455d-9889-c8606d5cc0f0 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:22:00.978130 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f623cf-e63c-4967-a95a-7bd51b416be1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:22:00.979014 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f623cf-e63c-4967-a95a-7bd51b416be1 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:22:00.980005 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f623cf-e63c-4967-a95a-7bd51b416be1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:22:00.980510 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f623cf-e63c-4967-a95a-7bd51b416be1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:22:00.982731 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f623cf-e63c-4967-a95a-7bd51b416be1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:22:00.984723 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f623cf-e63c-4967-a95a-7bd51b416be1 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:22:00.997115 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1349/2697] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Jul 25 12:22:00 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:22:01.001786 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd29371-03e6-4b5d-bda8-8ccb52415e1e admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:22:01.015323 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd29371-03e6-4b5d-bda8-8ccb52415e1e admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:22:01.020792 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fa2450-0449-4542-82ed-cc1bbdfa3bfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:22:01.021512 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fa2450-0449-4542-82ed-cc1bbdfa3bfc admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:22:01.021796 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fa2450-0449-4542-82ed-cc1bbdfa3bfc admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:22:01.022079 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fa2450-0449-4542-82ed-cc1bbdfa3bfc admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:22:01.022876 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fa2450-0449-4542-82ed-cc1bbdfa3bfc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:22:01.024775 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fa2450-0449-4542-82ed-cc1bbdfa3bfc admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:22:01.039593 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1349/2698] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Jul 25 12:22:01 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:22:01.041912 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f623cf-e63c-4967-a95a-7bd51b416be1 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:22:01.056025 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f623cf-e63c-4967-a95a-7bd51b416be1 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:22:01.061478 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f704e9-700a-409f-b73c-f7225b467f6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:22:01.062611 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f704e9-700a-409f-b73c-f7225b467f6f admin admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:22:01.063026 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f704e9-700a-409f-b73c-f7225b467f6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:22:01.064404 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f704e9-700a-409f-b73c-f7225b467f6f admin admin] PATH_INFO: `/v3/projects/116fc3d0402d4aa5a42223ceb5c10e69/users/f4bd83771e454d76bb6b29d497096f57/roles/472313ae69174578824df0bed6390744` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:22:01.079535 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f704e9-700a-409f-b73c-f7225b467f6f admin admin] RBAC: Authorizing `identity:create_grant(project_id=116fc3d0402d4aa5a42223ceb5c10e69, user_id=f4bd83771e454d76bb6b29d497096f57, role_id=472313ae69174578824df0bed6390744)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:22:01.082069 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f704e9-700a-409f-b73c-f7225b467f6f admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:22:01.099485 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1350/2699] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Jul 25 12:22:01 2022] PUT /identity/v3/projects/116fc3d0402d4aa5a42223ceb5c10e69/users/f4bd83771e454d76bb6b29d497096f57/roles/472313ae69174578824df0bed6390744 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:22:01.109950 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07b3617-5e60-4ecf-b226-cf643c33f40e None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:22:01.111931 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07b3617-5e60-4ecf-b226-cf643c33f40e None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:22:01.112324 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07b3617-5e60-4ecf-b226-cf643c33f40e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:22:01.149117 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-e07b3617-5e60-4ecf-b226-cf643c33f40e None None] MFA Rules not processed for user `f4bd83771e454d76bb6b29d497096f57`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:22:01.150400 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-e07b3617-5e60-4ecf-b226-cf643c33f40e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:22:01.236162 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1350/2700] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:22:01 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:22:01.241327 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f704e9-700a-409f-b73c-f7225b467f6f admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:22:01.322848 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f704e9-700a-409f-b73c-f7225b467f6f admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:22:01.328202 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396dac2e-5e5f-437f-8ee5-277244658b3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:22:01.329172 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396dac2e-5e5f-437f-8ee5-277244658b3d admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:22:01.329619 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396dac2e-5e5f-437f-8ee5-277244658b3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:22:01.330068 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396dac2e-5e5f-437f-8ee5-277244658b3d admin admin] PATH_INFO: `/v3/projects` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:22:01.330788 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396dac2e-5e5f-437f-8ee5-277244658b3d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:22:01.332770 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396dac2e-5e5f-437f-8ee5-277244658b3d admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:22:01.379292 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1351/2701] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Jul 25 12:22:01 2022] POST /identity/v3/projects => generated 373 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 12:22:01.381137 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07b3617-5e60-4ecf-b226-cf643c33f40e None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:22:01.391847 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07b3617-5e60-4ecf-b226-cf643c33f40e None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:22:01.422510 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:22:01.455955 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94d15c8-939e-41d0-b828-e7069fea07da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:22:01.456893 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94d15c8-939e-41d0-b828-e7069fea07da admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:22:01.457235 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94d15c8-939e-41d0-b828-e7069fea07da admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:22:01.457717 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94d15c8-939e-41d0-b828-e7069fea07da admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:22:01.458474 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94d15c8-939e-41d0-b828-e7069fea07da admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:22:01.461346 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94d15c8-939e-41d0-b828-e7069fea07da admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:22:01.496929 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1351/2702] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Jul 25 12:22:01 2022] POST /identity/v3/users => generated 293 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:22:01.498195 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396dac2e-5e5f-437f-8ee5-277244658b3d admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:22:01.518221 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396dac2e-5e5f-437f-8ee5-277244658b3d admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:22:01.524313 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58451f04-8c8e-4c6a-9304-52889c236abd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:22:01.525508 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58451f04-8c8e-4c6a-9304-52889c236abd admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:22:01.526021 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58451f04-8c8e-4c6a-9304-52889c236abd admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:22:01.526500 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58451f04-8c8e-4c6a-9304-52889c236abd admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:22:01.527610 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58451f04-8c8e-4c6a-9304-52889c236abd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:22:01.530053 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58451f04-8c8e-4c6a-9304-52889c236abd admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:22:01.550685 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1352/2703] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Jul 25 12:22:01 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:22:01.552468 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94d15c8-939e-41d0-b828-e7069fea07da admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:22:01.563893 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94d15c8-939e-41d0-b828-e7069fea07da admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:22:01.569001 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fc96f9-8597-490d-866a-0be15c65010f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:22:01.570032 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fc96f9-8597-490d-866a-0be15c65010f admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:22:01.570393 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fc96f9-8597-490d-866a-0be15c65010f admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:22:01.570880 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fc96f9-8597-490d-866a-0be15c65010f admin admin] PATH_INFO: `/v3/projects/192c0bfd0e25499ba551635ec141a9de/users/9a76a40109bf43b3b79e613c98ca1fe9/roles/a7d20885b6d94928a34b3cc5b3a00006` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:22:01.596618 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fc96f9-8597-490d-866a-0be15c65010f admin admin] RBAC: Authorizing `identity:create_grant(project_id=192c0bfd0e25499ba551635ec141a9de, user_id=9a76a40109bf43b3b79e613c98ca1fe9, role_id=a7d20885b6d94928a34b3cc5b3a00006)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:22:01.606744 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fc96f9-8597-490d-866a-0be15c65010f admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:22:01.628303 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1352/2704] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Jul 25 12:22:01 2022] PUT /identity/v3/projects/192c0bfd0e25499ba551635ec141a9de/users/9a76a40109bf43b3b79e613c98ca1fe9/roles/a7d20885b6d94928a34b3cc5b3a00006 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:22:01.634329 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58451f04-8c8e-4c6a-9304-52889c236abd admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:22:01.649608 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58451f04-8c8e-4c6a-9304-52889c236abd admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:22:01.654433 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:22:01.724223 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd83da1-4808-4014-990f-e65d7f41ce4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:22:01.724902 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd83da1-4808-4014-990f-e65d7f41ce4e admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:22:01.725187 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd83da1-4808-4014-990f-e65d7f41ce4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:22:01.725466 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd83da1-4808-4014-990f-e65d7f41ce4e admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:22:01.726982 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd83da1-4808-4014-990f-e65d7f41ce4e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:22:01.728693 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd83da1-4808-4014-990f-e65d7f41ce4e admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:22:01.742602 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1353/2705] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Jul 25 12:22:01 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:22:01.744375 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fc96f9-8597-490d-866a-0be15c65010f admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:22:01.755831 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fc96f9-8597-490d-866a-0be15c65010f admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:22:01.763040 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a9aa94-e6b0-49b7-80c2-ff1ef4676259 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:22:01.763990 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a9aa94-e6b0-49b7-80c2-ff1ef4676259 admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:22:01.764272 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a9aa94-e6b0-49b7-80c2-ff1ef4676259 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:22:01.764618 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a9aa94-e6b0-49b7-80c2-ff1ef4676259 admin admin] PATH_INFO: `/v3/projects/192c0bfd0e25499ba551635ec141a9de/users/9a76a40109bf43b3b79e613c98ca1fe9/roles/a7d20885b6d94928a34b3cc5b3a00006` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:22:01.767869 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a9aa94-e6b0-49b7-80c2-ff1ef4676259 admin admin] RBAC: Authorizing `identity:create_grant(project_id=192c0bfd0e25499ba551635ec141a9de, user_id=9a76a40109bf43b3b79e613c98ca1fe9, role_id=a7d20885b6d94928a34b3cc5b3a00006)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:22:01.772062 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a9aa94-e6b0-49b7-80c2-ff1ef4676259 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:22:01.791515 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1353/2706] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Jul 25 12:22:01 2022] PUT /identity/v3/projects/192c0bfd0e25499ba551635ec141a9de/users/9a76a40109bf43b3b79e613c98ca1fe9/roles/a7d20885b6d94928a34b3cc5b3a00006 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 12:22:01.803947 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9fa2a6-6738-49d0-824d-c52ab35f1429 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:22:01.805027 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9fa2a6-6738-49d0-824d-c52ab35f1429 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:22:01.805269 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9fa2a6-6738-49d0-824d-c52ab35f1429 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:22:01.838723 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-cf9fa2a6-6738-49d0-824d-c52ab35f1429 None None] MFA Rules not processed for user `9a76a40109bf43b3b79e613c98ca1fe9`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:22:01.839998 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-cf9fa2a6-6738-49d0-824d-c52ab35f1429 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:22:01.926168 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1354/2707] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:22:01 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:22:01.933485 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a9aa94-e6b0-49b7-80c2-ff1ef4676259 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:22:01.937568 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-72a9aa94-e6b0-49b7-80c2-ff1ef4676259 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:22:02.034973 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a9aa94-e6b0-49b7-80c2-ff1ef4676259 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:22:02.041306 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78303f62-41f7-4211-9d1e-bb54b1d20813 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f89ce6f0cf04373a414d927fa0f8a7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:22:02.042858 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78303f62-41f7-4211-9d1e-bb54b1d20813 service cinder] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:22:02.042858 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78303f62-41f7-4211-9d1e-bb54b1d20813 service cinder] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:22:02.043341 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78303f62-41f7-4211-9d1e-bb54b1d20813 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:22:02.055887 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78303f62-41f7-4211-9d1e-bb54b1d20813 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:22:02.057462 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78303f62-41f7-4211-9d1e-bb54b1d20813 service cinder] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:22:02.080989 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1354/2708] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:22:01 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:22:02.188527 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9fa2a6-6738-49d0-824d-c52ab35f1429 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:22:02.190542 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-cf9fa2a6-6738-49d0-824d-c52ab35f1429 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:22:02.210506 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9fa2a6-6738-49d0-824d-c52ab35f1429 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:22:02.214865 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155f6620-d8be-47df-a775-c34eceae4fc3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f89ce6f0cf04373a414d927fa0f8a7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:22:02.216036 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155f6620-d8be-47df-a775-c34eceae4fc3 service cinder] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:22:02.216553 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155f6620-d8be-47df-a775-c34eceae4fc3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:22:02.217000 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155f6620-d8be-47df-a775-c34eceae4fc3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:22:02.297109 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155f6620-d8be-47df-a775-c34eceae4fc3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:22:02.299984 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155f6620-d8be-47df-a775-c34eceae4fc3 service cinder] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:22:02.311636 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1355/2709] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:22:02 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:22:12.064228 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=56,keystone:INSERT=5 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:22:12.307215 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=88 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:22:15.842034 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6558991d-8efd-4bf6-91ed-d8afa57f6597 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:22:15.842514 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6558991d-8efd-4bf6-91ed-d8afa57f6597 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:22:15.842723 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6558991d-8efd-4bf6-91ed-d8afa57f6597 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:22:15.862010 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-6558991d-8efd-4bf6-91ed-d8afa57f6597 None None] MFA Rules not processed for user `93207b858d294d049fa56d45810a40aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:22:15.863208 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-6558991d-8efd-4bf6-91ed-d8afa57f6597 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:22:15.917813 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1355/2710] 10.222.0.42 () {58 vars in 1009 bytes} [Mon Jul 25 12:22:15 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:22:15.962664 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155f6620-d8be-47df-a775-c34eceae4fc3 service cinder] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:22:15.964261 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-155f6620-d8be-47df-a775-c34eceae4fc3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:22:16.035242 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155f6620-d8be-47df-a775-c34eceae4fc3 service cinder] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:22:16.040345 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2534b937-1762-4dbc-a830-e070b3596837 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bd782ea66364eeb96592096eae63d4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:22:16.040999 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2534b937-1762-4dbc-a830-e070b3596837 service swift] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:22:16.041289 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2534b937-1762-4dbc-a830-e070b3596837 service swift] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:22:16.041572 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2534b937-1762-4dbc-a830-e070b3596837 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:22:16.049507 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2534b937-1762-4dbc-a830-e070b3596837 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:22:16.051079 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2534b937-1762-4dbc-a830-e070b3596837 service swift] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:22:16.059134 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1356/2711] 10.222.0.42 () {60 vars in 1384 bytes} [Mon Jul 25 12:22:15 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 25 12:22:16.603042 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa112b3-6ebf-4971-93de-bfd0e6bebda0 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:22:16.603457 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa112b3-6ebf-4971-93de-bfd0e6bebda0 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:22:16.604631 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa112b3-6ebf-4971-93de-bfd0e6bebda0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:22:16.631511 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-6aa112b3-6ebf-4971-93de-bfd0e6bebda0 None None] MFA Rules not processed for user `93207b858d294d049fa56d45810a40aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:22:16.633640 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-6aa112b3-6ebf-4971-93de-bfd0e6bebda0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:22:16.642559 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1356/2712] 10.222.0.42 () {58 vars in 967 bytes} [Mon Jul 25 12:22:16 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 12:22:16.658236 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2534b937-1762-4dbc-a830-e070b3596837 service swift] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:22:16.669900 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2534b937-1762-4dbc-a830-e070b3596837 service swift] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:22:16.674961 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d11cec-5485-4896-a4c0-94c4bbf5edfb service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bd782ea66364eeb96592096eae63d4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:22:16.675626 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d11cec-5485-4896-a4c0-94c4bbf5edfb service swift] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:22:16.675946 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d11cec-5485-4896-a4c0-94c4bbf5edfb service swift] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:22:16.676223 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d11cec-5485-4896-a4c0-94c4bbf5edfb service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:22:16.685076 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d11cec-5485-4896-a4c0-94c4bbf5edfb service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:22:16.686824 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d11cec-5485-4896-a4c0-94c4bbf5edfb service swift] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:22:16.695431 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1357/2713] 10.222.0.42 () {60 vars in 1384 bytes} [Mon Jul 25 12:22:16 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 25 12:22:16.910777 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038f6464-7501-4cba-8456-41ea91954a50 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:22:16.911233 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038f6464-7501-4cba-8456-41ea91954a50 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:22:16.911459 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038f6464-7501-4cba-8456-41ea91954a50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:22:16.931373 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-038f6464-7501-4cba-8456-41ea91954a50 None None] MFA Rules not processed for user `93207b858d294d049fa56d45810a40aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:22:16.932663 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-038f6464-7501-4cba-8456-41ea91954a50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:22:16.940260 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1357/2714] 10.222.0.42 () {58 vars in 1009 bytes} [Mon Jul 25 12:22:16 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 12:22:16.954731 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d11cec-5485-4896-a4c0-94c4bbf5edfb service swift] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:22:16.966809 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d11cec-5485-4896-a4c0-94c4bbf5edfb service swift] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:22:16.972492 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b888ce-160e-465f-81d5-344efc60b043 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bd782ea66364eeb96592096eae63d4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:22:16.973532 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b888ce-160e-465f-81d5-344efc60b043 service swift] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:22:16.973928 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b888ce-160e-465f-81d5-344efc60b043 service swift] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:22:16.975011 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b888ce-160e-465f-81d5-344efc60b043 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:22:16.984086 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b888ce-160e-465f-81d5-344efc60b043 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:22:16.985655 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b888ce-160e-465f-81d5-344efc60b043 service swift] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:22:16.994165 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1358/2715] 10.222.0.42 () {60 vars in 1384 bytes} [Mon Jul 25 12:22:16 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 25 12:22:17.308639 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9203f23f-c9fd-435c-bdc2-488247e7b708 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:22:17.308931 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9203f23f-c9fd-435c-bdc2-488247e7b708 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:22:17.309210 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9203f23f-c9fd-435c-bdc2-488247e7b708 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:22:17.328169 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-9203f23f-c9fd-435c-bdc2-488247e7b708 None None] MFA Rules not processed for user `93207b858d294d049fa56d45810a40aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:22:17.329377 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-9203f23f-c9fd-435c-bdc2-488247e7b708 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:22:17.337035 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1358/2716] 10.222.0.42 () {58 vars in 967 bytes} [Mon Jul 25 12:22:17 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:22:17.348931 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b888ce-160e-465f-81d5-344efc60b043 service swift] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:22:17.360257 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b888ce-160e-465f-81d5-344efc60b043 service swift] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:22:17.365708 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb156fbb-7a15-4a7f-aefb-6108ba88d455 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bd782ea66364eeb96592096eae63d4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:22:17.366454 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb156fbb-7a15-4a7f-aefb-6108ba88d455 service swift] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:22:17.366750 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb156fbb-7a15-4a7f-aefb-6108ba88d455 service swift] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:22:17.367036 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb156fbb-7a15-4a7f-aefb-6108ba88d455 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:22:17.375992 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb156fbb-7a15-4a7f-aefb-6108ba88d455 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:22:17.377758 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb156fbb-7a15-4a7f-aefb-6108ba88d455 service swift] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:22:17.386864 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1359/2717] 10.222.0.42 () {60 vars in 1384 bytes} [Mon Jul 25 12:22:17 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 25 12:22:19.516979 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9203f23f-c9fd-435c-bdc2-488247e7b708 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:22:19.518397 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-9203f23f-c9fd-435c-bdc2-488247e7b708 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:22:19.588059 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9203f23f-c9fd-435c-bdc2-488247e7b708 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:22:19.592559 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24195b78-d8e2-415d-8989-95660c68c70a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:22:19.593427 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24195b78-d8e2-415d-8989-95660c68c70a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:22:19.593715 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24195b78-d8e2-415d-8989-95660c68c70a admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:22:19.593998 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24195b78-d8e2-415d-8989-95660c68c70a admin admin] PATH_INFO: `/v3/users/922702e951594912bea9546b33ed0335` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:22:19.595205 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24195b78-d8e2-415d-8989-95660c68c70a admin admin] RBAC: Authorizing `identity:delete_user(user_id=922702e951594912bea9546b33ed0335)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:22:19.597187 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24195b78-d8e2-415d-8989-95660c68c70a admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:22:19.642576 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-24195b78-d8e2-415d-8989-95660c68c70a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '922702e951594912bea9546b33ed0335'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:22:19.649844 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-24195b78-d8e2-415d-8989-95660c68c70a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '922702e951594912bea9546b33ed0335'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:22:19.656723 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-24195b78-d8e2-415d-8989-95660c68c70a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '922702e951594912bea9546b33ed0335'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:22:19.668800 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-24195b78-d8e2-415d-8989-95660c68c70a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '922702e951594912bea9546b33ed0335'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:22:19.669446 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-24195b78-d8e2-415d-8989-95660c68c70a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '922702e951594912bea9546b33ed0335'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:22:19.675298 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1359/2718] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Jul 25 12:22:19 2022] DELETE /identity/v3/users/922702e951594912bea9546b33ed0335 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:22:19.680633 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5302fa1-774f-4a4a-9d38-ebd9f6c7406b None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:22:19.681145 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5302fa1-774f-4a4a-9d38-ebd9f6c7406b None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:22:19.681245 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5302fa1-774f-4a4a-9d38-ebd9f6c7406b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:22:19.700436 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-c5302fa1-774f-4a4a-9d38-ebd9f6c7406b None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:22:19.701646 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-c5302fa1-774f-4a4a-9d38-ebd9f6c7406b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:22:19.759710 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1360/2719] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:22:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:22:19.770935 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24195b78-d8e2-415d-8989-95660c68c70a admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:22:19.772670 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-24195b78-d8e2-415d-8989-95660c68c70a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:22:19.779787 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:22:19.831732 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24195b78-d8e2-415d-8989-95660c68c70a admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:22:19.835759 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c29c3ac-066e-4bf4-b127-c2197bb6a924 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51733781201243e8abf9eca39b207750', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:22:19.836528 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c29c3ac-066e-4bf4-b127-c2197bb6a924 service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:22:19.836821 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c29c3ac-066e-4bf4-b127-c2197bb6a924 service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:22:19.837102 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c29c3ac-066e-4bf4-b127-c2197bb6a924 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:22:19.846279 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c29c3ac-066e-4bf4-b127-c2197bb6a924 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:22:19.847757 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c29c3ac-066e-4bf4-b127-c2197bb6a924 service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:22:19.857763 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1360/2720] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:22:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:22:20.025777 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5302fa1-774f-4a4a-9d38-ebd9f6c7406b None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:22:20.027412 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-c5302fa1-774f-4a4a-9d38-ebd9f6c7406b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:22:20.041541 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5302fa1-774f-4a4a-9d38-ebd9f6c7406b None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:22:20.046322 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb42f1f-0fd9-4090-8ad4-67dac1370c63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:22:20.047322 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb42f1f-0fd9-4090-8ad4-67dac1370c63 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:22:20.047642 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb42f1f-0fd9-4090-8ad4-67dac1370c63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:22:20.047942 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb42f1f-0fd9-4090-8ad4-67dac1370c63 admin admin] PATH_INFO: `/v3/projects/f6b1d49d16c741d195afa089f5f1eb36` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:22:20.049160 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb42f1f-0fd9-4090-8ad4-67dac1370c63 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f6b1d49d16c741d195afa089f5f1eb36)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:22:20.051071 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb42f1f-0fd9-4090-8ad4-67dac1370c63 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:22:20.096264 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:22:20.099875 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-6bb42f1f-0fd9-4090-8ad4-67dac1370c63 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f6b1d49d16c741d195afa089f5f1eb36', 'request_id': 'req-6bb42f1f-0fd9-4090-8ad4-67dac1370c63'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:22:20.108555 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-6bb42f1f-0fd9-4090-8ad4-67dac1370c63 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f6b1d49d16c741d195afa089f5f1eb36', 'request_id': 'req-6bb42f1f-0fd9-4090-8ad4-67dac1370c63'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:22:20.115390 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-6bb42f1f-0fd9-4090-8ad4-67dac1370c63 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f6b1d49d16c741d195afa089f5f1eb36', 'request_id': 'req-6bb42f1f-0fd9-4090-8ad4-67dac1370c63'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:22:20.127489 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-6bb42f1f-0fd9-4090-8ad4-67dac1370c63 admin admin] The token cache is being invalidate because project f6b1d49d16c741d195afa089f5f1eb36 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80298) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 12:22:20.127805 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-6bb42f1f-0fd9-4090-8ad4-67dac1370c63 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:22:20.128440 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-6bb42f1f-0fd9-4090-8ad4-67dac1370c63 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:22:20.129983 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1361/2721] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Jul 25 12:22:20 2022] DELETE /identity/v3/projects/f6b1d49d16c741d195afa089f5f1eb36 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:22:25.243509 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdf98f1-73bc-4490-9e1e-21123308aed1 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:22:25.244007 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdf98f1-73bc-4490-9e1e-21123308aed1 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:22:25.244104 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdf98f1-73bc-4490-9e1e-21123308aed1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:22:25.263524 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-cbdf98f1-73bc-4490-9e1e-21123308aed1 None None] MFA Rules not processed for user `3494ce75df62497bb737254fd1665a3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:22:25.264760 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-cbdf98f1-73bc-4490-9e1e-21123308aed1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:22:25.318020 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1361/2722] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:22:25 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:22:25.640257 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb42f1f-0fd9-4090-8ad4-67dac1370c63 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:22:25.642024 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-6bb42f1f-0fd9-4090-8ad4-67dac1370c63 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:22:25.700585 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb42f1f-0fd9-4090-8ad4-67dac1370c63 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:22:25.704178 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40a4b8f-1703-40b1-821f-69b0ae65afbf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51733781201243e8abf9eca39b207750', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:22:25.704914 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40a4b8f-1703-40b1-821f-69b0ae65afbf service neutron] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:22:25.705206 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40a4b8f-1703-40b1-821f-69b0ae65afbf service neutron] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:22:25.705488 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40a4b8f-1703-40b1-821f-69b0ae65afbf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:22:25.707328 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-d40a4b8f-1703-40b1-821f-69b0ae65afbf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:22:25.717247 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40a4b8f-1703-40b1-821f-69b0ae65afbf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:22:25.718792 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40a4b8f-1703-40b1-821f-69b0ae65afbf service neutron] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:22:25.727946 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1362/2723] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:22:25 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:22:27.816981 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdf98f1-73bc-4490-9e1e-21123308aed1 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:22:27.818867 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-cbdf98f1-73bc-4490-9e1e-21123308aed1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:22:27.881957 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdf98f1-73bc-4490-9e1e-21123308aed1 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:22:27.886455 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4110e51-3159-4e7f-a839-9d7f2ad70c37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:22:27.887310 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4110e51-3159-4e7f-a839-9d7f2ad70c37 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:22:27.887622 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4110e51-3159-4e7f-a839-9d7f2ad70c37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:22:27.887904 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4110e51-3159-4e7f-a839-9d7f2ad70c37 admin admin] PATH_INFO: `/v3/users/a170b4b5faed40f188ce6751904f109a` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:22:27.889065 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4110e51-3159-4e7f-a839-9d7f2ad70c37 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a170b4b5faed40f188ce6751904f109a)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:22:27.891002 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4110e51-3159-4e7f-a839-9d7f2ad70c37 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:22:27.937482 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-b4110e51-3159-4e7f-a839-9d7f2ad70c37 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a170b4b5faed40f188ce6751904f109a'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:22:27.942701 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:22:27.945503 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-b4110e51-3159-4e7f-a839-9d7f2ad70c37 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a170b4b5faed40f188ce6751904f109a'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:22:27.953497 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-b4110e51-3159-4e7f-a839-9d7f2ad70c37 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a170b4b5faed40f188ce6751904f109a'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:22:27.966777 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-b4110e51-3159-4e7f-a839-9d7f2ad70c37 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a170b4b5faed40f188ce6751904f109a'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:22:27.967528 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-b4110e51-3159-4e7f-a839-9d7f2ad70c37 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a170b4b5faed40f188ce6751904f109a'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:22:27.973774 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1362/2724] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Jul 25 12:22:27 2022] DELETE /identity/v3/users/a170b4b5faed40f188ce6751904f109a => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:22:27.978444 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb28770b-d858-40e8-8de8-341c1951481b None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:22:27.978907 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb28770b-d858-40e8-8de8-341c1951481b None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:22:27.979052 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb28770b-d858-40e8-8de8-341c1951481b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:22:27.997760 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-cb28770b-d858-40e8-8de8-341c1951481b None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:22:27.998968 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-cb28770b-d858-40e8-8de8-341c1951481b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:22:28.056745 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1363/2725] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:22:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:22:28.066563 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4110e51-3159-4e7f-a839-9d7f2ad70c37 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:22:28.067994 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-b4110e51-3159-4e7f-a839-9d7f2ad70c37 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:22:28.125590 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4110e51-3159-4e7f-a839-9d7f2ad70c37 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:22:28.131857 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65223042-312d-482d-9a3b-104f7ae776da service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51733781201243e8abf9eca39b207750', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:22:28.132638 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65223042-312d-482d-9a3b-104f7ae776da service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:22:28.133253 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65223042-312d-482d-9a3b-104f7ae776da service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:22:28.133683 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65223042-312d-482d-9a3b-104f7ae776da service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:22:28.144109 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65223042-312d-482d-9a3b-104f7ae776da service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:22:28.145716 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65223042-312d-482d-9a3b-104f7ae776da service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:22:28.155397 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1363/2726] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:22:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:22:28.455743 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb28770b-d858-40e8-8de8-341c1951481b None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:22:28.457247 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-cb28770b-d858-40e8-8de8-341c1951481b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:22:28.472426 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb28770b-d858-40e8-8de8-341c1951481b None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:22:28.478712 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e23c4d-86ac-46f9-bd6f-c59323b2bd44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:22:28.479653 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e23c4d-86ac-46f9-bd6f-c59323b2bd44 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:22:28.479946 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e23c4d-86ac-46f9-bd6f-c59323b2bd44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:22:28.480285 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e23c4d-86ac-46f9-bd6f-c59323b2bd44 admin admin] PATH_INFO: `/v3/projects/d7bc9b29547841d8b46f264148eae000` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:22:28.481436 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e23c4d-86ac-46f9-bd6f-c59323b2bd44 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d7bc9b29547841d8b46f264148eae000)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:22:28.483352 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e23c4d-86ac-46f9-bd6f-c59323b2bd44 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:22:28.533192 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-36e23c4d-86ac-46f9-bd6f-c59323b2bd44 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd7bc9b29547841d8b46f264148eae000', 'request_id': 'req-36e23c4d-86ac-46f9-bd6f-c59323b2bd44'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:22:28.540994 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-36e23c4d-86ac-46f9-bd6f-c59323b2bd44 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd7bc9b29547841d8b46f264148eae000', 'request_id': 'req-36e23c4d-86ac-46f9-bd6f-c59323b2bd44'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:22:28.546678 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-36e23c4d-86ac-46f9-bd6f-c59323b2bd44 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd7bc9b29547841d8b46f264148eae000', 'request_id': 'req-36e23c4d-86ac-46f9-bd6f-c59323b2bd44'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:22:28.554924 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:DELETE=11,keystone:SELECT=87,keystone:INSERT=2 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:22:28.559335 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-36e23c4d-86ac-46f9-bd6f-c59323b2bd44 admin admin] The token cache is being invalidate because project d7bc9b29547841d8b46f264148eae000 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80298) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 12:22:28.560070 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-36e23c4d-86ac-46f9-bd6f-c59323b2bd44 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:22:28.560963 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-36e23c4d-86ac-46f9-bd6f-c59323b2bd44 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:22:28.562930 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1364/2727] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Jul 25 12:22:28 2022] DELETE /identity/v3/projects/d7bc9b29547841d8b46f264148eae000 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:22:28.568046 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535210cf-93d6-40c1-9fee-7a363f98fd16 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:22:28.568337 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535210cf-93d6-40c1-9fee-7a363f98fd16 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:22:28.568621 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535210cf-93d6-40c1-9fee-7a363f98fd16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:22:28.588310 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-535210cf-93d6-40c1-9fee-7a363f98fd16 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:22:28.589530 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-535210cf-93d6-40c1-9fee-7a363f98fd16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:22:28.648745 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1364/2728] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:22:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:22:28.656386 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c157aa30-a653-4701-a243-f666f30fc07c None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:22:28.656685 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c157aa30-a653-4701-a243-f666f30fc07c None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:22:28.656972 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c157aa30-a653-4701-a243-f666f30fc07c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:22:28.677042 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-c157aa30-a653-4701-a243-f666f30fc07c None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:22:28.678858 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-c157aa30-a653-4701-a243-f666f30fc07c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:22:28.686852 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1365/2729] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:22:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:22:28.691286 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535210cf-93d6-40c1-9fee-7a363f98fd16 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:22:28.700888 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535210cf-93d6-40c1-9fee-7a363f98fd16 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:22:28.705367 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97709e4-5413-4a76-b242-53effe7dc700 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:22:28.706010 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97709e4-5413-4a76-b242-53effe7dc700 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:22:28.706295 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97709e4-5413-4a76-b242-53effe7dc700 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:22:28.706576 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97709e4-5413-4a76-b242-53effe7dc700 admin admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:22:28.707298 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97709e4-5413-4a76-b242-53effe7dc700 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:22:28.707636 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97709e4-5413-4a76-b242-53effe7dc700 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:22:28.709236 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97709e4-5413-4a76-b242-53effe7dc700 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:22:28.720553 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1365/2730] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Jul 25 12:22:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 12:22:28.725058 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c157aa30-a653-4701-a243-f666f30fc07c None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:22:28.734000 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c157aa30-a653-4701-a243-f666f30fc07c None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:22:28.738618 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aabef1d-e691-4ebd-bc28-665801511d11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:22:28.739260 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aabef1d-e691-4ebd-bc28-665801511d11 admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:22:28.739561 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aabef1d-e691-4ebd-bc28-665801511d11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:22:28.739854 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aabef1d-e691-4ebd-bc28-665801511d11 admin admin] PATH_INFO: `/v3/projects` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:22:28.741186 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aabef1d-e691-4ebd-bc28-665801511d11 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:22:28.743088 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aabef1d-e691-4ebd-bc28-665801511d11 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:22:28.775166 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1366/2731] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Jul 25 12:22:28 2022] POST /identity/v3/projects => generated 369 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 12:22:28.776663 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97709e4-5413-4a76-b242-53effe7dc700 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:22:28.790577 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97709e4-5413-4a76-b242-53effe7dc700 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:22:28.846690 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30aba62d-6342-4795-b77b-9b1d7c267eb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:22:28.847343 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30aba62d-6342-4795-b77b-9b1d7c267eb7 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:22:28.847579 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30aba62d-6342-4795-b77b-9b1d7c267eb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:22:28.847853 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30aba62d-6342-4795-b77b-9b1d7c267eb7 admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:22:28.848535 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30aba62d-6342-4795-b77b-9b1d7c267eb7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:22:28.851099 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30aba62d-6342-4795-b77b-9b1d7c267eb7 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:22:28.859745 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:22:28.872817 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1366/2732] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Jul 25 12:22:28 2022] POST /identity/v3/users => generated 285 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:22:28.877127 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aabef1d-e691-4ebd-bc28-665801511d11 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:22:28.887620 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aabef1d-e691-4ebd-bc28-665801511d11 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:22:28.892351 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4c677d-4647-47aa-8439-1bc74c166565 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:22:28.893072 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4c677d-4647-47aa-8439-1bc74c166565 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:22:28.893363 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4c677d-4647-47aa-8439-1bc74c166565 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:22:28.893649 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4c677d-4647-47aa-8439-1bc74c166565 admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:22:28.894377 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4c677d-4647-47aa-8439-1bc74c166565 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:22:28.896006 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4c677d-4647-47aa-8439-1bc74c166565 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:22:28.906602 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1367/2733] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Jul 25 12:22:28 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:22:28.910346 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30aba62d-6342-4795-b77b-9b1d7c267eb7 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:22:28.920811 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30aba62d-6342-4795-b77b-9b1d7c267eb7 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:22:28.925993 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0b1f14-6ca4-43a3-8b36-043e3c961e3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:22:28.926711 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0b1f14-6ca4-43a3-8b36-043e3c961e3c admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:22:28.927004 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0b1f14-6ca4-43a3-8b36-043e3c961e3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:22:28.927321 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0b1f14-6ca4-43a3-8b36-043e3c961e3c admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:22:28.928194 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0b1f14-6ca4-43a3-8b36-043e3c961e3c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:22:28.929809 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0b1f14-6ca4-43a3-8b36-043e3c961e3c admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:22:28.940365 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1367/2734] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Jul 25 12:22:28 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:22:28.944407 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4c677d-4647-47aa-8439-1bc74c166565 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:22:28.954914 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4c677d-4647-47aa-8439-1bc74c166565 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:22:28.960159 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366e6a63-90e2-43d8-8a7d-a9f08922c382 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:22:28.961030 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366e6a63-90e2-43d8-8a7d-a9f08922c382 admin admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:22:28.961317 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366e6a63-90e2-43d8-8a7d-a9f08922c382 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:22:28.961596 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366e6a63-90e2-43d8-8a7d-a9f08922c382 admin admin] PATH_INFO: `/v3/projects/19f4b84bef1848c9ae54d90ad5cd08b5/users/5d0c7419a7554fbc928fa88d8cf13ceb/roles/472313ae69174578824df0bed6390744` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:22:28.976678 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366e6a63-90e2-43d8-8a7d-a9f08922c382 admin admin] RBAC: Authorizing `identity:create_grant(project_id=19f4b84bef1848c9ae54d90ad5cd08b5, user_id=5d0c7419a7554fbc928fa88d8cf13ceb, role_id=472313ae69174578824df0bed6390744)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:22:28.979371 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366e6a63-90e2-43d8-8a7d-a9f08922c382 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:22:28.992046 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1368/2735] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Jul 25 12:22:28 2022] PUT /identity/v3/projects/19f4b84bef1848c9ae54d90ad5cd08b5/users/5d0c7419a7554fbc928fa88d8cf13ceb/roles/472313ae69174578824df0bed6390744 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:22:29.002576 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7bc3be-33c3-4b80-859a-d4fc584f0e2e None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:22:29.002874 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7bc3be-33c3-4b80-859a-d4fc584f0e2e None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:22:29.003157 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7bc3be-33c3-4b80-859a-d4fc584f0e2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:22:29.032226 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-fb7bc3be-33c3-4b80-859a-d4fc584f0e2e None None] MFA Rules not processed for user `5d0c7419a7554fbc928fa88d8cf13ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:22:29.033422 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-fb7bc3be-33c3-4b80-859a-d4fc584f0e2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:22:29.231654 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1368/2736] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:22:29 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:22:29.242881 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366e6a63-90e2-43d8-8a7d-a9f08922c382 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:22:29.244573 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-366e6a63-90e2-43d8-8a7d-a9f08922c382 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:22:29.302191 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366e6a63-90e2-43d8-8a7d-a9f08922c382 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:22:29.305842 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1d0ecc-c350-4c22-b3c5-6fbe0dde599d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ea16528b1374bae9cf54603f7907bfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:22:29.306478 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1d0ecc-c350-4c22-b3c5-6fbe0dde599d service glance] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:22:29.306759 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1d0ecc-c350-4c22-b3c5-6fbe0dde599d service glance] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:22:29.307031 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1d0ecc-c350-4c22-b3c5-6fbe0dde599d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:22:29.315224 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1d0ecc-c350-4c22-b3c5-6fbe0dde599d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:22:29.316670 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1d0ecc-c350-4c22-b3c5-6fbe0dde599d service glance] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:22:29.325892 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1369/2737] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:22:29 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:22:39.228688 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:INSERT=2,keystone:SELECT=39 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:22:39.321890 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=53,keystone:INSERT=2 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:22:43.121872 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7bc3be-33c3-4b80-859a-d4fc584f0e2e None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:22:43.123300 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-fb7bc3be-33c3-4b80-859a-d4fc584f0e2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:22:43.246832 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7bc3be-33c3-4b80-859a-d4fc584f0e2e None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:22:43.251223 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e74225-2d8e-413b-8f58-f95c35dfae3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:22:43.252105 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e74225-2d8e-413b-8f58-f95c35dfae3c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:22:43.252388 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e74225-2d8e-413b-8f58-f95c35dfae3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:22:43.252664 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e74225-2d8e-413b-8f58-f95c35dfae3c admin admin] PATH_INFO: `/v3/users/f4bd83771e454d76bb6b29d497096f57` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:22:43.253937 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e74225-2d8e-413b-8f58-f95c35dfae3c admin admin] RBAC: Authorizing `identity:delete_user(user_id=f4bd83771e454d76bb6b29d497096f57)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:22:43.255864 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e74225-2d8e-413b-8f58-f95c35dfae3c admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:22:43.300184 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-21e74225-2d8e-413b-8f58-f95c35dfae3c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f4bd83771e454d76bb6b29d497096f57'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:22:43.307323 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-21e74225-2d8e-413b-8f58-f95c35dfae3c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f4bd83771e454d76bb6b29d497096f57'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:22:43.314711 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-21e74225-2d8e-413b-8f58-f95c35dfae3c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f4bd83771e454d76bb6b29d497096f57'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:22:43.328383 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-21e74225-2d8e-413b-8f58-f95c35dfae3c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f4bd83771e454d76bb6b29d497096f57'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:22:43.329282 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-21e74225-2d8e-413b-8f58-f95c35dfae3c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f4bd83771e454d76bb6b29d497096f57'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:22:43.335442 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1369/2738] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Jul 25 12:22:43 2022] DELETE /identity/v3/users/f4bd83771e454d76bb6b29d497096f57 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:22:43.340275 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15144bc7-b297-4e74-9880-d1ba90647b79 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:22:43.340741 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15144bc7-b297-4e74-9880-d1ba90647b79 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:22:43.340841 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15144bc7-b297-4e74-9880-d1ba90647b79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:22:43.367936 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-15144bc7-b297-4e74-9880-d1ba90647b79 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:22:43.369150 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-15144bc7-b297-4e74-9880-d1ba90647b79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:22:43.426569 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1370/2739] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:22:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:22:43.437230 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e74225-2d8e-413b-8f58-f95c35dfae3c admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:22:43.438820 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-21e74225-2d8e-413b-8f58-f95c35dfae3c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:22:43.459773 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:22:43.524993 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e74225-2d8e-413b-8f58-f95c35dfae3c admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:22:43.528668 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ad70db-eec7-4f3d-b18d-4866e4f5e8fb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51733781201243e8abf9eca39b207750', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:22:43.529316 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ad70db-eec7-4f3d-b18d-4866e4f5e8fb service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:22:43.529605 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ad70db-eec7-4f3d-b18d-4866e4f5e8fb service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:22:43.529887 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ad70db-eec7-4f3d-b18d-4866e4f5e8fb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:22:43.539343 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ad70db-eec7-4f3d-b18d-4866e4f5e8fb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:22:43.540792 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ad70db-eec7-4f3d-b18d-4866e4f5e8fb service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:22:43.550082 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1370/2740] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:22:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:22:44.065598 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15144bc7-b297-4e74-9880-d1ba90647b79 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:22:44.067086 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-15144bc7-b297-4e74-9880-d1ba90647b79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:22:44.080346 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15144bc7-b297-4e74-9880-d1ba90647b79 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:22:44.085997 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae105d32-c4f2-488d-b4ec-b18bfda9a012 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:22:44.086901 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae105d32-c4f2-488d-b4ec-b18bfda9a012 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:22:44.087192 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae105d32-c4f2-488d-b4ec-b18bfda9a012 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:22:44.087473 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae105d32-c4f2-488d-b4ec-b18bfda9a012 admin admin] PATH_INFO: `/v3/projects/116fc3d0402d4aa5a42223ceb5c10e69` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:22:44.088835 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae105d32-c4f2-488d-b4ec-b18bfda9a012 admin admin] RBAC: Authorizing `identity:delete_project(project_id=116fc3d0402d4aa5a42223ceb5c10e69)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:22:44.090769 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae105d32-c4f2-488d-b4ec-b18bfda9a012 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:22:44.152604 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-ae105d32-c4f2-488d-b4ec-b18bfda9a012 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '116fc3d0402d4aa5a42223ceb5c10e69', 'request_id': 'req-ae105d32-c4f2-488d-b4ec-b18bfda9a012'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:22:44.159872 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-ae105d32-c4f2-488d-b4ec-b18bfda9a012 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '116fc3d0402d4aa5a42223ceb5c10e69', 'request_id': 'req-ae105d32-c4f2-488d-b4ec-b18bfda9a012'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:22:44.165496 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-ae105d32-c4f2-488d-b4ec-b18bfda9a012 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '116fc3d0402d4aa5a42223ceb5c10e69', 'request_id': 'req-ae105d32-c4f2-488d-b4ec-b18bfda9a012'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:22:44.176410 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-ae105d32-c4f2-488d-b4ec-b18bfda9a012 admin admin] The token cache is being invalidate because project 116fc3d0402d4aa5a42223ceb5c10e69 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80298) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 12:22:44.176709 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-ae105d32-c4f2-488d-b4ec-b18bfda9a012 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:22:44.177346 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-ae105d32-c4f2-488d-b4ec-b18bfda9a012 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:22:44.178881 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1371/2741] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Jul 25 12:22:44 2022] DELETE /identity/v3/projects/116fc3d0402d4aa5a42223ceb5c10e69 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:22:44.182995 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ad70db-eec7-4f3d-b18d-4866e4f5e8fb service neutron] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:22:44.192523 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ad70db-eec7-4f3d-b18d-4866e4f5e8fb service neutron] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:22:44.246603 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0701e0-3d40-4e74-9fff-486d13dcf9a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:22:44.247643 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0701e0-3d40-4e74-9fff-486d13dcf9a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:22:44.247796 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0701e0-3d40-4e74-9fff-486d13dcf9a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:22:44.248046 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0701e0-3d40-4e74-9fff-486d13dcf9a4 admin admin] PATH_INFO: `/v3/users/9a76a40109bf43b3b79e613c98ca1fe9` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:22:44.249234 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0701e0-3d40-4e74-9fff-486d13dcf9a4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9a76a40109bf43b3b79e613c98ca1fe9)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:22:44.251272 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0701e0-3d40-4e74-9fff-486d13dcf9a4 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:22:44.297468 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-6c0701e0-3d40-4e74-9fff-486d13dcf9a4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9a76a40109bf43b3b79e613c98ca1fe9'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:22:44.304696 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-6c0701e0-3d40-4e74-9fff-486d13dcf9a4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9a76a40109bf43b3b79e613c98ca1fe9'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:22:44.311892 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-6c0701e0-3d40-4e74-9fff-486d13dcf9a4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9a76a40109bf43b3b79e613c98ca1fe9'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:22:44.324098 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-6c0701e0-3d40-4e74-9fff-486d13dcf9a4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9a76a40109bf43b3b79e613c98ca1fe9'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:22:44.324728 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-6c0701e0-3d40-4e74-9fff-486d13dcf9a4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9a76a40109bf43b3b79e613c98ca1fe9'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:22:44.330584 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1371/2742] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Jul 25 12:22:44 2022] DELETE /identity/v3/users/9a76a40109bf43b3b79e613c98ca1fe9 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:22:44.697906 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae105d32-c4f2-488d-b4ec-b18bfda9a012 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:22:44.700148 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-ae105d32-c4f2-488d-b4ec-b18bfda9a012 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:22:44.729865 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae105d32-c4f2-488d-b4ec-b18bfda9a012 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:22:44.797883 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f362ee1-3232-4908-9cfa-59a74ebb3499 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:22:44.798789 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f362ee1-3232-4908-9cfa-59a74ebb3499 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:22:44.799082 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f362ee1-3232-4908-9cfa-59a74ebb3499 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:22:44.799366 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f362ee1-3232-4908-9cfa-59a74ebb3499 admin admin] PATH_INFO: `/v3/projects/192c0bfd0e25499ba551635ec141a9de` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:22:44.800621 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f362ee1-3232-4908-9cfa-59a74ebb3499 admin admin] RBAC: Authorizing `identity:delete_project(project_id=192c0bfd0e25499ba551635ec141a9de)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:22:44.802564 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f362ee1-3232-4908-9cfa-59a74ebb3499 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:22:44.826816 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:22:44.854386 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-4f362ee1-3232-4908-9cfa-59a74ebb3499 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '192c0bfd0e25499ba551635ec141a9de', 'request_id': 'req-4f362ee1-3232-4908-9cfa-59a74ebb3499'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:22:44.862681 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-4f362ee1-3232-4908-9cfa-59a74ebb3499 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '192c0bfd0e25499ba551635ec141a9de', 'request_id': 'req-4f362ee1-3232-4908-9cfa-59a74ebb3499'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:22:44.868955 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-4f362ee1-3232-4908-9cfa-59a74ebb3499 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '192c0bfd0e25499ba551635ec141a9de', 'request_id': 'req-4f362ee1-3232-4908-9cfa-59a74ebb3499'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:22:44.880269 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-4f362ee1-3232-4908-9cfa-59a74ebb3499 admin admin] The token cache is being invalidate because project 192c0bfd0e25499ba551635ec141a9de was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80298) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 12:22:44.880562 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-4f362ee1-3232-4908-9cfa-59a74ebb3499 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:22:44.881871 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-4f362ee1-3232-4908-9cfa-59a74ebb3499 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:22:44.883433 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1372/2743] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Jul 25 12:22:44 2022] DELETE /identity/v3/projects/192c0bfd0e25499ba551635ec141a9de => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:22:44.888216 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefcb2f8-04fc-469e-bec5-3c0fa32cd125 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:22:44.888506 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefcb2f8-04fc-469e-bec5-3c0fa32cd125 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:22:44.888787 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefcb2f8-04fc-469e-bec5-3c0fa32cd125 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:22:44.921000 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:22:44.938238 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-cefcb2f8-04fc-469e-bec5-3c0fa32cd125 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:22:44.939438 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-cefcb2f8-04fc-469e-bec5-3c0fa32cd125 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:22:44.999981 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1372/2744] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:22:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:22:45.008179 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7fef2b-cdc7-4a2c-98fb-0d89ba3cb582 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:22:45.008476 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7fef2b-cdc7-4a2c-98fb-0d89ba3cb582 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:22:45.008766 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7fef2b-cdc7-4a2c-98fb-0d89ba3cb582 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:22:45.029632 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-0a7fef2b-cdc7-4a2c-98fb-0d89ba3cb582 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:22:45.030848 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-0a7fef2b-cdc7-4a2c-98fb-0d89ba3cb582 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:22:45.039385 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1373/2745] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:22:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:22:45.043639 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefcb2f8-04fc-469e-bec5-3c0fa32cd125 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:22:45.053935 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefcb2f8-04fc-469e-bec5-3c0fa32cd125 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:22:45.058545 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c295fdc-8ae8-41fe-8899-edf2538e55c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:22:45.059200 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c295fdc-8ae8-41fe-8899-edf2538e55c1 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:22:45.059485 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c295fdc-8ae8-41fe-8899-edf2538e55c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:22:45.059793 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c295fdc-8ae8-41fe-8899-edf2538e55c1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:22:45.060523 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c295fdc-8ae8-41fe-8899-edf2538e55c1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:22:45.060832 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c295fdc-8ae8-41fe-8899-edf2538e55c1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:22:45.062457 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c295fdc-8ae8-41fe-8899-edf2538e55c1 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:22:45.078366 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1373/2746] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Jul 25 12:22:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:22:45.079614 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7fef2b-cdc7-4a2c-98fb-0d89ba3cb582 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:22:45.090960 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7fef2b-cdc7-4a2c-98fb-0d89ba3cb582 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:22:45.095706 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a92bd4-bf8f-43ff-b21f-ef0305910dff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:22:45.096430 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a92bd4-bf8f-43ff-b21f-ef0305910dff admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:22:45.096639 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a92bd4-bf8f-43ff-b21f-ef0305910dff admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:22:45.096923 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a92bd4-bf8f-43ff-b21f-ef0305910dff admin admin] PATH_INFO: `/v3/projects` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:22:45.097615 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a92bd4-bf8f-43ff-b21f-ef0305910dff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:22:45.099523 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a92bd4-bf8f-43ff-b21f-ef0305910dff admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:22:45.128483 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1374/2747] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Jul 25 12:22:45 2022] POST /identity/v3/projects => generated 355 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:22:45.132748 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c295fdc-8ae8-41fe-8899-edf2538e55c1 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:22:45.142452 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c295fdc-8ae8-41fe-8899-edf2538e55c1 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:22:45.197730 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420983b9-34d5-44ff-a87f-a2368eed42fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:22:45.198413 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420983b9-34d5-44ff-a87f-a2368eed42fe admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:22:45.198697 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420983b9-34d5-44ff-a87f-a2368eed42fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:22:45.198976 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420983b9-34d5-44ff-a87f-a2368eed42fe admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:22:45.199691 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420983b9-34d5-44ff-a87f-a2368eed42fe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:22:45.201580 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420983b9-34d5-44ff-a87f-a2368eed42fe admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:22:45.221973 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1374/2748] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Jul 25 12:22:45 2022] POST /identity/v3/users => generated 278 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:22:45.226557 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a92bd4-bf8f-43ff-b21f-ef0305910dff admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:22:45.236625 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a92bd4-bf8f-43ff-b21f-ef0305910dff admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:22:45.241808 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681a4d48-f029-476b-8b40-6fd78500b871 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:22:45.242460 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681a4d48-f029-476b-8b40-6fd78500b871 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:22:45.242747 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681a4d48-f029-476b-8b40-6fd78500b871 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:22:45.243024 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681a4d48-f029-476b-8b40-6fd78500b871 admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:22:45.243778 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681a4d48-f029-476b-8b40-6fd78500b871 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:22:45.245366 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681a4d48-f029-476b-8b40-6fd78500b871 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:22:45.255635 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1375/2749] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Jul 25 12:22:45 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:22:45.259409 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420983b9-34d5-44ff-a87f-a2368eed42fe admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:22:45.269402 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420983b9-34d5-44ff-a87f-a2368eed42fe admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:22:45.274875 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f1b324-66ad-45c6-bf70-b823c7d0b831 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:22:45.275533 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f1b324-66ad-45c6-bf70-b823c7d0b831 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:22:45.275848 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f1b324-66ad-45c6-bf70-b823c7d0b831 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:22:45.276129 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f1b324-66ad-45c6-bf70-b823c7d0b831 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:22:45.276945 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f1b324-66ad-45c6-bf70-b823c7d0b831 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:22:45.278675 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f1b324-66ad-45c6-bf70-b823c7d0b831 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:22:45.289927 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1375/2750] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Jul 25 12:22:45 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:22:45.294020 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681a4d48-f029-476b-8b40-6fd78500b871 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:22:45.303865 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681a4d48-f029-476b-8b40-6fd78500b871 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:22:45.308438 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5e1d21-d50d-4698-ab7e-bc2def18ff64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:22:45.309296 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5e1d21-d50d-4698-ab7e-bc2def18ff64 admin admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:22:45.309582 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5e1d21-d50d-4698-ab7e-bc2def18ff64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:22:45.309861 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5e1d21-d50d-4698-ab7e-bc2def18ff64 admin admin] PATH_INFO: `/v3/projects/0f8cf01606c545a7aa585ce26e01fd5a/users/36e35159f1ad48cdb3a9b43e8178365c/roles/472313ae69174578824df0bed6390744` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:22:45.324286 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5e1d21-d50d-4698-ab7e-bc2def18ff64 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0f8cf01606c545a7aa585ce26e01fd5a, user_id=36e35159f1ad48cdb3a9b43e8178365c, role_id=472313ae69174578824df0bed6390744)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:22:45.326941 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5e1d21-d50d-4698-ab7e-bc2def18ff64 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:22:45.340235 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1376/2751] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Jul 25 12:22:45 2022] PUT /identity/v3/projects/0f8cf01606c545a7aa585ce26e01fd5a/users/36e35159f1ad48cdb3a9b43e8178365c/roles/472313ae69174578824df0bed6390744 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 12:22:45.352321 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e5124c-26d8-469f-b4f5-971671b057d4 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:22:45.352637 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e5124c-26d8-469f-b4f5-971671b057d4 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:22:45.352918 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e5124c-26d8-469f-b4f5-971671b057d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:22:45.379438 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-21e5124c-26d8-469f-b4f5-971671b057d4 None None] MFA Rules not processed for user `36e35159f1ad48cdb3a9b43e8178365c`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:22:45.380703 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-21e5124c-26d8-469f-b4f5-971671b057d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:22:45.434381 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:22:45.444085 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1376/2752] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:22:45 2022] POST /identity/v3/auth/tokens => generated 3222 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:22:45.448551 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5e1d21-d50d-4698-ab7e-bc2def18ff64 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:22:45.508155 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5e1d21-d50d-4698-ab7e-bc2def18ff64 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:22:45.512710 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26608eea-345d-4bc7-809b-7d04bc32bcc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:22:45.513351 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26608eea-345d-4bc7-809b-7d04bc32bcc5 admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:22:45.513637 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26608eea-345d-4bc7-809b-7d04bc32bcc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:22:45.513919 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26608eea-345d-4bc7-809b-7d04bc32bcc5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:22:45.514683 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26608eea-345d-4bc7-809b-7d04bc32bcc5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:22:45.516600 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26608eea-345d-4bc7-809b-7d04bc32bcc5 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:22:45.543123 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1377/2753] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Jul 25 12:22:45 2022] POST /identity/v3/projects => generated 355 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:22:45.547267 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e5124c-26d8-469f-b4f5-971671b057d4 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:22:45.558168 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e5124c-26d8-469f-b4f5-971671b057d4 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:22:45.612546 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebbe1c0-7d1f-4c48-b879-b441a77a9fbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:22:45.613132 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebbe1c0-7d1f-4c48-b879-b441a77a9fbe admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:22:45.613420 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebbe1c0-7d1f-4c48-b879-b441a77a9fbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:22:45.613700 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebbe1c0-7d1f-4c48-b879-b441a77a9fbe admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:22:45.614389 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebbe1c0-7d1f-4c48-b879-b441a77a9fbe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:22:45.616312 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebbe1c0-7d1f-4c48-b879-b441a77a9fbe admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:22:45.636948 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1377/2754] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Jul 25 12:22:45 2022] POST /identity/v3/users => generated 284 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:22:45.641005 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26608eea-345d-4bc7-809b-7d04bc32bcc5 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:22:45.652925 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26608eea-345d-4bc7-809b-7d04bc32bcc5 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:22:45.657514 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fa0a97-c3ca-4ec8-b351-6be2ac147c50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:22:45.658172 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fa0a97-c3ca-4ec8-b351-6be2ac147c50 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:22:45.658457 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fa0a97-c3ca-4ec8-b351-6be2ac147c50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:22:45.658737 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fa0a97-c3ca-4ec8-b351-6be2ac147c50 admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:22:45.659466 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76fa0a97-c3ca-4ec8-b351-6be2ac147c50 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:22:45.661092 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76fa0a97-c3ca-4ec8-b351-6be2ac147c50 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:22:45.673878 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1378/2755] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Jul 25 12:22:45 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:22:45.678140 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebbe1c0-7d1f-4c48-b879-b441a77a9fbe admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:22:45.688529 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebbe1c0-7d1f-4c48-b879-b441a77a9fbe admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:22:45.693261 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df87ec3-7cb1-457b-b601-86eee5c2fe59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:22:45.694955 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df87ec3-7cb1-457b-b601-86eee5c2fe59 admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:22:45.695460 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df87ec3-7cb1-457b-b601-86eee5c2fe59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:22:45.696013 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df87ec3-7cb1-457b-b601-86eee5c2fe59 admin admin] PATH_INFO: `/v3/projects/5984833842854f5e90646f8b9f32cb63/users/9a2ba62524e544e89fdd9d99033b75e1/roles/a7d20885b6d94928a34b3cc5b3a00006` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:22:45.712577 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df87ec3-7cb1-457b-b601-86eee5c2fe59 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5984833842854f5e90646f8b9f32cb63, user_id=9a2ba62524e544e89fdd9d99033b75e1, role_id=a7d20885b6d94928a34b3cc5b3a00006)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:22:45.715197 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df87ec3-7cb1-457b-b601-86eee5c2fe59 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:22:45.728325 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1378/2756] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Jul 25 12:22:45 2022] PUT /identity/v3/projects/5984833842854f5e90646f8b9f32cb63/users/9a2ba62524e544e89fdd9d99033b75e1/roles/a7d20885b6d94928a34b3cc5b3a00006 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:22:45.733052 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fa0a97-c3ca-4ec8-b351-6be2ac147c50 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:22:45.746288 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fa0a97-c3ca-4ec8-b351-6be2ac147c50 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:22:45.760101 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:22:45.824717 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c374b6ce-b133-43e4-9bd1-b58c7d116520 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:22:45.825402 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c374b6ce-b133-43e4-9bd1-b58c7d116520 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:22:45.825711 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c374b6ce-b133-43e4-9bd1-b58c7d116520 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:22:45.825992 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c374b6ce-b133-43e4-9bd1-b58c7d116520 admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:22:45.826715 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c374b6ce-b133-43e4-9bd1-b58c7d116520 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:22:45.828346 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c374b6ce-b133-43e4-9bd1-b58c7d116520 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:22:45.839370 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1379/2757] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Jul 25 12:22:45 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:22:45.844739 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df87ec3-7cb1-457b-b601-86eee5c2fe59 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:22:45.855903 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df87ec3-7cb1-457b-b601-86eee5c2fe59 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:22:45.864996 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0694cdcb-3072-4a63-b068-4f9b9a73c938 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:22:45.866105 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0694cdcb-3072-4a63-b068-4f9b9a73c938 admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:22:45.866526 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0694cdcb-3072-4a63-b068-4f9b9a73c938 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:22:45.866900 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0694cdcb-3072-4a63-b068-4f9b9a73c938 admin admin] PATH_INFO: `/v3/projects/5984833842854f5e90646f8b9f32cb63/users/9a2ba62524e544e89fdd9d99033b75e1/roles/a7d20885b6d94928a34b3cc5b3a00006` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:22:45.874433 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0694cdcb-3072-4a63-b068-4f9b9a73c938 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5984833842854f5e90646f8b9f32cb63, user_id=9a2ba62524e544e89fdd9d99033b75e1, role_id=a7d20885b6d94928a34b3cc5b3a00006)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:22:45.879488 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0694cdcb-3072-4a63-b068-4f9b9a73c938 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:22:45.893476 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1379/2758] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Jul 25 12:22:45 2022] PUT /identity/v3/projects/5984833842854f5e90646f8b9f32cb63/users/9a2ba62524e544e89fdd9d99033b75e1/roles/a7d20885b6d94928a34b3cc5b3a00006 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:22:45.904449 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92b872a-40bc-46f4-b145-e00eb38393f7 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:22:45.904748 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92b872a-40bc-46f4-b145-e00eb38393f7 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:22:45.905039 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92b872a-40bc-46f4-b145-e00eb38393f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:22:45.933039 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-a92b872a-40bc-46f4-b145-e00eb38393f7 None None] MFA Rules not processed for user `9a2ba62524e544e89fdd9d99033b75e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:22:45.934262 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-a92b872a-40bc-46f4-b145-e00eb38393f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:22:46.009036 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1380/2759] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:22:45 2022] POST /identity/v3/auth/tokens => generated 3289 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:22:46.016472 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0694cdcb-3072-4a63-b068-4f9b9a73c938 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:22:46.018450 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-0694cdcb-3072-4a63-b068-4f9b9a73c938 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:22:46.096087 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0694cdcb-3072-4a63-b068-4f9b9a73c938 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:22:46.099851 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb367e26-6fb2-40a3-901d-7a79006a546b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f89ce6f0cf04373a414d927fa0f8a7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:22:46.100504 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb367e26-6fb2-40a3-901d-7a79006a546b service cinder] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:22:46.100796 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb367e26-6fb2-40a3-901d-7a79006a546b service cinder] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:22:46.101081 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb367e26-6fb2-40a3-901d-7a79006a546b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:22:46.110806 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb367e26-6fb2-40a3-901d-7a79006a546b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:22:46.112293 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb367e26-6fb2-40a3-901d-7a79006a546b service cinder] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:22:46.125891 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1380/2760] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:22:46 2022] GET /identity/v3/auth/tokens => generated 3289 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:22:46.221888 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92b872a-40bc-46f4-b145-e00eb38393f7 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:22:46.231862 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92b872a-40bc-46f4-b145-e00eb38393f7 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:22:46.235656 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb785f9-59c4-4abc-a2d9-fdc093c3a454 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f89ce6f0cf04373a414d927fa0f8a7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:22:46.236385 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb785f9-59c4-4abc-a2d9-fdc093c3a454 service cinder] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:22:46.236748 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb785f9-59c4-4abc-a2d9-fdc093c3a454 service cinder] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:22:46.237102 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb785f9-59c4-4abc-a2d9-fdc093c3a454 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:22:46.296792 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb785f9-59c4-4abc-a2d9-fdc093c3a454 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:22:46.298253 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb785f9-59c4-4abc-a2d9-fdc093c3a454 service cinder] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:22:46.308754 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1381/2761] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:22:46 2022] GET /identity/v3/auth/tokens => generated 3222 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:22:46.510746 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb367e26-6fb2-40a3-901d-7a79006a546b service cinder] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:22:46.512181 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-eb367e26-6fb2-40a3-901d-7a79006a546b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:22:46.567501 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb367e26-6fb2-40a3-901d-7a79006a546b service cinder] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:22:46.571064 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbc7bb0-0789-4420-a6e7-141e26ea97ed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51733781201243e8abf9eca39b207750', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:22:46.571730 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbc7bb0-0789-4420-a6e7-141e26ea97ed service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:22:46.572018 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbc7bb0-0789-4420-a6e7-141e26ea97ed service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:22:46.572432 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbc7bb0-0789-4420-a6e7-141e26ea97ed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:22:46.574238 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-acbc7bb0-0789-4420-a6e7-141e26ea97ed service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:22:46.579002 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:22:46.636869 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbc7bb0-0789-4420-a6e7-141e26ea97ed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:22:46.638229 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbc7bb0-0789-4420-a6e7-141e26ea97ed service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:22:46.648709 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1381/2762] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:22:46 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:22:48.262432 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb785f9-59c4-4abc-a2d9-fdc093c3a454 service cinder] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:22:48.263901 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-bdb785f9-59c4-4abc-a2d9-fdc093c3a454 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:22:48.307538 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:22:48.368008 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb785f9-59c4-4abc-a2d9-fdc093c3a454 service cinder] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:22:48.373630 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d260803c-c28b-40f0-8145-a326fda4d5db service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f67142a1193d47c7bd33d7040268233c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:22:48.374283 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d260803c-c28b-40f0-8145-a326fda4d5db service nova] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:22:48.374576 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d260803c-c28b-40f0-8145-a326fda4d5db service nova] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:22:48.374857 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d260803c-c28b-40f0-8145-a326fda4d5db service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:22:48.376644 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-d260803c-c28b-40f0-8145-a326fda4d5db service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:22:48.388885 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d260803c-c28b-40f0-8145-a326fda4d5db service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:22:48.390457 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d260803c-c28b-40f0-8145-a326fda4d5db service nova] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:22:48.402099 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1382/2763] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:22:48 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:22:50.701789 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbc7bb0-0789-4420-a6e7-141e26ea97ed service neutron] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:22:50.703240 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-acbc7bb0-0789-4420-a6e7-141e26ea97ed service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:22:50.795821 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbc7bb0-0789-4420-a6e7-141e26ea97ed service neutron] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:22:50.800927 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bc711d-dd3c-41b7-9758-c612504b5a9f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8012afab6cac4358ab878732e58d0689', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:22:50.801580 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bc711d-dd3c-41b7-9758-c612504b5a9f service placement] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:22:50.801869 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bc711d-dd3c-41b7-9758-c612504b5a9f service placement] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:22:50.802155 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bc711d-dd3c-41b7-9758-c612504b5a9f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:22:50.804012 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-e1bc711d-dd3c-41b7-9758-c612504b5a9f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:22:50.815928 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bc711d-dd3c-41b7-9758-c612504b5a9f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:22:50.817547 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bc711d-dd3c-41b7-9758-c612504b5a9f service placement] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:22:50.827562 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1382/2764] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:22:50 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:22:55.148028 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d260803c-c28b-40f0-8145-a326fda4d5db service nova] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:22:55.149511 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-d260803c-c28b-40f0-8145-a326fda4d5db service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:22:55.213784 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d260803c-c28b-40f0-8145-a326fda4d5db service nova] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:22:55.218367 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da53975-ea0b-4700-9f46-d4213c6a9396 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:22:55.219232 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da53975-ea0b-4700-9f46-d4213c6a9396 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:22:55.219520 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da53975-ea0b-4700-9f46-d4213c6a9396 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:22:55.219837 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da53975-ea0b-4700-9f46-d4213c6a9396 admin admin] PATH_INFO: `/v3/users/5d0c7419a7554fbc928fa88d8cf13ceb` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:22:55.221020 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da53975-ea0b-4700-9f46-d4213c6a9396 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5d0c7419a7554fbc928fa88d8cf13ceb)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:22:55.222922 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da53975-ea0b-4700-9f46-d4213c6a9396 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:22:55.268470 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-7da53975-ea0b-4700-9f46-d4213c6a9396 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5d0c7419a7554fbc928fa88d8cf13ceb'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:22:55.269119 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-7da53975-ea0b-4700-9f46-d4213c6a9396 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5d0c7419a7554fbc928fa88d8cf13ceb'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:22:55.277154 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-7da53975-ea0b-4700-9f46-d4213c6a9396 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5d0c7419a7554fbc928fa88d8cf13ceb'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:22:55.290157 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-7da53975-ea0b-4700-9f46-d4213c6a9396 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5d0c7419a7554fbc928fa88d8cf13ceb'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:22:55.297350 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-7da53975-ea0b-4700-9f46-d4213c6a9396 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5d0c7419a7554fbc928fa88d8cf13ceb'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:22:55.303685 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1383/2765] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Jul 25 12:22:55 2022] DELETE /identity/v3/users/5d0c7419a7554fbc928fa88d8cf13ceb => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:22:55.307971 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea08ae93-51af-4414-af1c-9c9944ed3462 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:22:55.308423 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea08ae93-51af-4414-af1c-9c9944ed3462 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:22:55.308717 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea08ae93-51af-4414-af1c-9c9944ed3462 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:22:55.328669 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-ea08ae93-51af-4414-af1c-9c9944ed3462 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:22:55.329936 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-ea08ae93-51af-4414-af1c-9c9944ed3462 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:22:55.390122 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1383/2766] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:22:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:22:55.400623 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da53975-ea0b-4700-9f46-d4213c6a9396 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:22:55.402757 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-7da53975-ea0b-4700-9f46-d4213c6a9396 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:22:55.422215 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:22:55.487312 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da53975-ea0b-4700-9f46-d4213c6a9396 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:22:55.491589 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abc8f64-8a36-454b-931e-f44f8ad9adc7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51733781201243e8abf9eca39b207750', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:22:55.492473 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abc8f64-8a36-454b-931e-f44f8ad9adc7 service neutron] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:22:55.492765 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abc8f64-8a36-454b-931e-f44f8ad9adc7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:22:55.493592 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abc8f64-8a36-454b-931e-f44f8ad9adc7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:22:55.503704 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abc8f64-8a36-454b-931e-f44f8ad9adc7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:22:55.505815 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abc8f64-8a36-454b-931e-f44f8ad9adc7 service neutron] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:22:55.516221 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1384/2767] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:22:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:22:55.799120 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea08ae93-51af-4414-af1c-9c9944ed3462 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:22:55.801003 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-ea08ae93-51af-4414-af1c-9c9944ed3462 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:22:55.816741 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea08ae93-51af-4414-af1c-9c9944ed3462 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:22:55.823387 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1b43d0-0a91-431d-8c6c-7f7fdb4bd897 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:22:55.824210 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1b43d0-0a91-431d-8c6c-7f7fdb4bd897 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:22:55.824490 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1b43d0-0a91-431d-8c6c-7f7fdb4bd897 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:22:55.824772 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1b43d0-0a91-431d-8c6c-7f7fdb4bd897 admin admin] PATH_INFO: `/v3/projects/19f4b84bef1848c9ae54d90ad5cd08b5` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:22:55.829750 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1b43d0-0a91-431d-8c6c-7f7fdb4bd897 admin admin] RBAC: Authorizing `identity:delete_project(project_id=19f4b84bef1848c9ae54d90ad5cd08b5)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:22:55.831719 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1b43d0-0a91-431d-8c6c-7f7fdb4bd897 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:22:55.838608 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:22:55.882755 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-bc1b43d0-0a91-431d-8c6c-7f7fdb4bd897 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '19f4b84bef1848c9ae54d90ad5cd08b5', 'request_id': 'req-bc1b43d0-0a91-431d-8c6c-7f7fdb4bd897'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:22:55.890598 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-bc1b43d0-0a91-431d-8c6c-7f7fdb4bd897 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '19f4b84bef1848c9ae54d90ad5cd08b5', 'request_id': 'req-bc1b43d0-0a91-431d-8c6c-7f7fdb4bd897'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:22:55.896503 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-bc1b43d0-0a91-431d-8c6c-7f7fdb4bd897 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '19f4b84bef1848c9ae54d90ad5cd08b5', 'request_id': 'req-bc1b43d0-0a91-431d-8c6c-7f7fdb4bd897'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:22:55.907385 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-bc1b43d0-0a91-431d-8c6c-7f7fdb4bd897 admin admin] The token cache is being invalidate because project 19f4b84bef1848c9ae54d90ad5cd08b5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 12:22:55.907711 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-bc1b43d0-0a91-431d-8c6c-7f7fdb4bd897 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:22:55.908351 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-bc1b43d0-0a91-431d-8c6c-7f7fdb4bd897 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:22:55.909871 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1384/2768] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Jul 25 12:22:55 2022] DELETE /identity/v3/projects/19f4b84bef1848c9ae54d90ad5cd08b5 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:22:55.914707 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f827448-c8cb-482d-a103-076db5ebc335 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:22:55.915003 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f827448-c8cb-482d-a103-076db5ebc335 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:22:55.915288 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f827448-c8cb-482d-a103-076db5ebc335 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:22:55.935057 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-9f827448-c8cb-482d-a103-076db5ebc335 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:22:55.936313 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-9f827448-c8cb-482d-a103-076db5ebc335 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:22:55.994099 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1385/2769] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:22:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:22:56.001989 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893fde64-751c-4ad6-8147-4daa181d2235 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:22:56.002290 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893fde64-751c-4ad6-8147-4daa181d2235 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:22:56.002576 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893fde64-751c-4ad6-8147-4daa181d2235 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:22:56.022563 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-893fde64-751c-4ad6-8147-4daa181d2235 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:22:56.023835 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-893fde64-751c-4ad6-8147-4daa181d2235 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:22:56.031881 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1385/2770] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:22:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:22:56.036003 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f827448-c8cb-482d-a103-076db5ebc335 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:22:56.045676 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f827448-c8cb-482d-a103-076db5ebc335 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:22:56.050276 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e91f93-7d5a-40af-8af8-991e9d8ea060 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:22:56.050929 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e91f93-7d5a-40af-8af8-991e9d8ea060 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:22:56.051223 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e91f93-7d5a-40af-8af8-991e9d8ea060 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:22:56.051509 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e91f93-7d5a-40af-8af8-991e9d8ea060 admin admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:22:56.052830 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e91f93-7d5a-40af-8af8-991e9d8ea060 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:22:56.053166 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e91f93-7d5a-40af-8af8-991e9d8ea060 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:22:56.054750 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e91f93-7d5a-40af-8af8-991e9d8ea060 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:22:56.066137 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1386/2771] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Jul 25 12:22:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 12:22:56.070770 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893fde64-751c-4ad6-8147-4daa181d2235 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:22:56.080015 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893fde64-751c-4ad6-8147-4daa181d2235 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:22:56.084535 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32a9f54-7c57-46d6-991f-be103d02f322 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:22:56.085166 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32a9f54-7c57-46d6-991f-be103d02f322 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:22:56.085452 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32a9f54-7c57-46d6-991f-be103d02f322 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:22:56.085733 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32a9f54-7c57-46d6-991f-be103d02f322 admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:22:56.086415 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32a9f54-7c57-46d6-991f-be103d02f322 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:22:56.088634 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32a9f54-7c57-46d6-991f-be103d02f322 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:22:56.115429 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1386/2772] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Jul 25 12:22:56 2022] POST /identity/v3/projects => generated 363 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 12:22:56.119470 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e91f93-7d5a-40af-8af8-991e9d8ea060 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:22:56.129535 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e91f93-7d5a-40af-8af8-991e9d8ea060 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:22:56.190711 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edd8608-286f-4491-9a39-3de44fc53125 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:22:56.191396 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edd8608-286f-4491-9a39-3de44fc53125 admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:22:56.191611 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edd8608-286f-4491-9a39-3de44fc53125 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:22:56.191884 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edd8608-286f-4491-9a39-3de44fc53125 admin admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:22:56.192595 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8edd8608-286f-4491-9a39-3de44fc53125 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:22:56.194560 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8edd8608-286f-4491-9a39-3de44fc53125 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:22:56.218022 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1387/2773] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Jul 25 12:22:56 2022] POST /identity/v3/users => generated 282 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 12:22:56.219383 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32a9f54-7c57-46d6-991f-be103d02f322 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:22:56.230795 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32a9f54-7c57-46d6-991f-be103d02f322 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:22:56.235436 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6415cab5-1d23-4482-ae8c-0cf1398359fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:22:56.236139 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6415cab5-1d23-4482-ae8c-0cf1398359fa admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:22:56.236424 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6415cab5-1d23-4482-ae8c-0cf1398359fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:22:56.236704 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6415cab5-1d23-4482-ae8c-0cf1398359fa admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:22:56.237434 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6415cab5-1d23-4482-ae8c-0cf1398359fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:22:56.239020 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6415cab5-1d23-4482-ae8c-0cf1398359fa admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:22:56.249845 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1387/2774] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Jul 25 12:22:56 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:22:56.254028 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edd8608-286f-4491-9a39-3de44fc53125 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:22:56.264111 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edd8608-286f-4491-9a39-3de44fc53125 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:22:56.268669 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941fa668-a54a-4e67-9c7e-3eabfc4befe6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:22:56.269322 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941fa668-a54a-4e67-9c7e-3eabfc4befe6 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:22:56.269608 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941fa668-a54a-4e67-9c7e-3eabfc4befe6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:22:56.269892 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941fa668-a54a-4e67-9c7e-3eabfc4befe6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:22:56.270610 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941fa668-a54a-4e67-9c7e-3eabfc4befe6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:22:56.272551 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941fa668-a54a-4e67-9c7e-3eabfc4befe6 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:22:56.283193 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1388/2775] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Jul 25 12:22:56 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:22:56.287698 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6415cab5-1d23-4482-ae8c-0cf1398359fa admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:22:56.297828 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6415cab5-1d23-4482-ae8c-0cf1398359fa admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:22:56.303243 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beae4a61-5819-4ab4-9943-682d9b83f48f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:22:56.304393 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beae4a61-5819-4ab4-9943-682d9b83f48f admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:22:56.304717 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beae4a61-5819-4ab4-9943-682d9b83f48f admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:22:56.305025 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beae4a61-5819-4ab4-9943-682d9b83f48f admin admin] PATH_INFO: `/v3/projects/ccb7de30a6914b489252b2f2ce54f1b9/users/4c754338e83f4f83931be7c576015cd4/roles/472313ae69174578824df0bed6390744` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:22:56.320305 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beae4a61-5819-4ab4-9943-682d9b83f48f admin admin] RBAC: Authorizing `identity:create_grant(project_id=ccb7de30a6914b489252b2f2ce54f1b9, user_id=4c754338e83f4f83931be7c576015cd4, role_id=472313ae69174578824df0bed6390744)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:22:56.322991 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beae4a61-5819-4ab4-9943-682d9b83f48f admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:22:56.336167 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1388/2776] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Jul 25 12:22:56 2022] PUT /identity/v3/projects/ccb7de30a6914b489252b2f2ce54f1b9/users/4c754338e83f4f83931be7c576015cd4/roles/472313ae69174578824df0bed6390744 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:22:56.346478 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dd65c1-2712-4e77-88ef-ffbcef0d8d64 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:22:56.346779 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dd65c1-2712-4e77-88ef-ffbcef0d8d64 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:22:56.347102 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dd65c1-2712-4e77-88ef-ffbcef0d8d64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:22:56.355796 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:22:56.377300 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-66dd65c1-2712-4e77-88ef-ffbcef0d8d64 None None] MFA Rules not processed for user `4c754338e83f4f83931be7c576015cd4`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:22:56.378514 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-66dd65c1-2712-4e77-88ef-ffbcef0d8d64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:22:56.442992 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1389/2777] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:22:56 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:22:56.454338 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beae4a61-5819-4ab4-9943-682d9b83f48f admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:22:56.456054 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-beae4a61-5819-4ab4-9943-682d9b83f48f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:22:56.515620 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beae4a61-5819-4ab4-9943-682d9b83f48f admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:22:56.521990 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d909592-f04d-4704-9611-8323f76f71bd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f89ce6f0cf04373a414d927fa0f8a7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:22:56.523069 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d909592-f04d-4704-9611-8323f76f71bd service cinder] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:22:56.523509 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d909592-f04d-4704-9611-8323f76f71bd service cinder] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:22:56.523989 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d909592-f04d-4704-9611-8323f76f71bd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:22:56.539998 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d909592-f04d-4704-9611-8323f76f71bd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:22:56.543845 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d909592-f04d-4704-9611-8323f76f71bd service cinder] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:22:56.553524 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1389/2778] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:22:56 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:23:02.256129 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66dd65c1-2712-4e77-88ef-ffbcef0d8d64 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:23:02.258012 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-66dd65c1-2712-4e77-88ef-ffbcef0d8d64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:23:02.325605 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66dd65c1-2712-4e77-88ef-ffbcef0d8d64 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:23:02.330298 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec112c5d-5eed-4031-a28c-7bc0a8be8784 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:23:02.331165 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec112c5d-5eed-4031-a28c-7bc0a8be8784 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:23:02.331452 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec112c5d-5eed-4031-a28c-7bc0a8be8784 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:23:02.331777 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec112c5d-5eed-4031-a28c-7bc0a8be8784 admin admin] PATH_INFO: `/v3/users/36e35159f1ad48cdb3a9b43e8178365c` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:23:02.332983 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec112c5d-5eed-4031-a28c-7bc0a8be8784 admin admin] RBAC: Authorizing `identity:delete_user(user_id=36e35159f1ad48cdb3a9b43e8178365c)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:23:02.335485 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec112c5d-5eed-4031-a28c-7bc0a8be8784 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:23:02.379553 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-ec112c5d-5eed-4031-a28c-7bc0a8be8784 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '36e35159f1ad48cdb3a9b43e8178365c'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:23:02.380232 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-ec112c5d-5eed-4031-a28c-7bc0a8be8784 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '36e35159f1ad48cdb3a9b43e8178365c'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:23:02.388006 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-ec112c5d-5eed-4031-a28c-7bc0a8be8784 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '36e35159f1ad48cdb3a9b43e8178365c'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:23:02.400769 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-ec112c5d-5eed-4031-a28c-7bc0a8be8784 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '36e35159f1ad48cdb3a9b43e8178365c'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:23:02.407606 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-ec112c5d-5eed-4031-a28c-7bc0a8be8784 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '36e35159f1ad48cdb3a9b43e8178365c'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:23:02.413465 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1390/2779] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Jul 25 12:23:02 2022] DELETE /identity/v3/users/36e35159f1ad48cdb3a9b43e8178365c => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:23:02.417972 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b215b576-74dc-4887-b515-57d27dce52c3 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:23:02.418437 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b215b576-74dc-4887-b515-57d27dce52c3 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:23:02.418535 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b215b576-74dc-4887-b515-57d27dce52c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:23:02.437030 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-b215b576-74dc-4887-b515-57d27dce52c3 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:23:02.438239 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-b215b576-74dc-4887-b515-57d27dce52c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:23:02.467112 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:23:02.498036 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1390/2780] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:23:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:23:02.508325 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec112c5d-5eed-4031-a28c-7bc0a8be8784 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:23:02.509743 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-ec112c5d-5eed-4031-a28c-7bc0a8be8784 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:23:02.524639 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:23:02.570475 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec112c5d-5eed-4031-a28c-7bc0a8be8784 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:23:02.574110 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ad7c1b-2c5c-4d64-8b31-28e068ad5a4a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51733781201243e8abf9eca39b207750', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:23:02.574756 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ad7c1b-2c5c-4d64-8b31-28e068ad5a4a service neutron] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:23:02.575046 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ad7c1b-2c5c-4d64-8b31-28e068ad5a4a service neutron] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:23:02.575328 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ad7c1b-2c5c-4d64-8b31-28e068ad5a4a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:23:02.584740 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ad7c1b-2c5c-4d64-8b31-28e068ad5a4a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:23:02.586148 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ad7c1b-2c5c-4d64-8b31-28e068ad5a4a service neutron] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:23:02.595745 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1391/2781] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:23:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:23:02.861301 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b215b576-74dc-4887-b515-57d27dce52c3 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:23:02.862769 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-b215b576-74dc-4887-b515-57d27dce52c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:23:02.875940 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b215b576-74dc-4887-b515-57d27dce52c3 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:23:02.880679 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4131935-eef1-467e-9935-16035de6c572 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:23:02.881542 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4131935-eef1-467e-9935-16035de6c572 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:23:02.881826 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4131935-eef1-467e-9935-16035de6c572 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:23:02.882101 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4131935-eef1-467e-9935-16035de6c572 admin admin] PATH_INFO: `/v3/projects/0f8cf01606c545a7aa585ce26e01fd5a` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:23:02.883273 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4131935-eef1-467e-9935-16035de6c572 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0f8cf01606c545a7aa585ce26e01fd5a)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:23:02.885238 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4131935-eef1-467e-9935-16035de6c572 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:23:02.932107 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-d4131935-eef1-467e-9935-16035de6c572 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0f8cf01606c545a7aa585ce26e01fd5a', 'request_id': 'req-d4131935-eef1-467e-9935-16035de6c572'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:23:02.939072 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-d4131935-eef1-467e-9935-16035de6c572 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0f8cf01606c545a7aa585ce26e01fd5a', 'request_id': 'req-d4131935-eef1-467e-9935-16035de6c572'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:23:02.944579 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-d4131935-eef1-467e-9935-16035de6c572 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0f8cf01606c545a7aa585ce26e01fd5a', 'request_id': 'req-d4131935-eef1-467e-9935-16035de6c572'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:23:02.955253 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-d4131935-eef1-467e-9935-16035de6c572 admin admin] The token cache is being invalidate because project 0f8cf01606c545a7aa585ce26e01fd5a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 12:23:02.955567 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-d4131935-eef1-467e-9935-16035de6c572 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:23:02.956195 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-d4131935-eef1-467e-9935-16035de6c572 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:23:02.957682 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1391/2782] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Jul 25 12:23:02 2022] DELETE /identity/v3/projects/0f8cf01606c545a7aa585ce26e01fd5a => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 12:23:02.961633 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ad7c1b-2c5c-4d64-8b31-28e068ad5a4a service neutron] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:23:02.971297 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ad7c1b-2c5c-4d64-8b31-28e068ad5a4a service neutron] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:23:03.026947 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2ab474-d1bb-4427-bb9a-b57d339bf769 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:23:03.027851 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2ab474-d1bb-4427-bb9a-b57d339bf769 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:23:03.028142 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2ab474-d1bb-4427-bb9a-b57d339bf769 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:23:03.028426 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2ab474-d1bb-4427-bb9a-b57d339bf769 admin admin] PATH_INFO: `/v3/users/9a2ba62524e544e89fdd9d99033b75e1` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:23:03.029624 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2ab474-d1bb-4427-bb9a-b57d339bf769 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9a2ba62524e544e89fdd9d99033b75e1)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:23:03.031555 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2ab474-d1bb-4427-bb9a-b57d339bf769 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:23:03.078022 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-fd2ab474-d1bb-4427-bb9a-b57d339bf769 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9a2ba62524e544e89fdd9d99033b75e1'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:23:03.078667 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-fd2ab474-d1bb-4427-bb9a-b57d339bf769 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9a2ba62524e544e89fdd9d99033b75e1'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:23:03.087164 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-fd2ab474-d1bb-4427-bb9a-b57d339bf769 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9a2ba62524e544e89fdd9d99033b75e1'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:23:03.099941 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-fd2ab474-d1bb-4427-bb9a-b57d339bf769 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9a2ba62524e544e89fdd9d99033b75e1'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:23:03.107202 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-fd2ab474-d1bb-4427-bb9a-b57d339bf769 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9a2ba62524e544e89fdd9d99033b75e1'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:23:03.113249 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1392/2783] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Jul 25 12:23:02 2022] DELETE /identity/v3/users/9a2ba62524e544e89fdd9d99033b75e1 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:23:03.402853 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4131935-eef1-467e-9935-16035de6c572 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:23:03.404584 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-d4131935-eef1-467e-9935-16035de6c572 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:23:03.420357 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4131935-eef1-467e-9935-16035de6c572 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:23:03.484135 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc052b55-7e7b-4b50-8ef2-f1ac511bc27e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:23:03.485028 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc052b55-7e7b-4b50-8ef2-f1ac511bc27e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:23:03.485315 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc052b55-7e7b-4b50-8ef2-f1ac511bc27e admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:23:03.485598 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc052b55-7e7b-4b50-8ef2-f1ac511bc27e admin admin] PATH_INFO: `/v3/projects/5984833842854f5e90646f8b9f32cb63` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:23:03.486791 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc052b55-7e7b-4b50-8ef2-f1ac511bc27e admin admin] RBAC: Authorizing `identity:delete_project(project_id=5984833842854f5e90646f8b9f32cb63)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:23:03.488725 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc052b55-7e7b-4b50-8ef2-f1ac511bc27e admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:23:03.534114 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-cc052b55-7e7b-4b50-8ef2-f1ac511bc27e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5984833842854f5e90646f8b9f32cb63', 'request_id': 'req-cc052b55-7e7b-4b50-8ef2-f1ac511bc27e'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:23:03.541300 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-cc052b55-7e7b-4b50-8ef2-f1ac511bc27e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5984833842854f5e90646f8b9f32cb63', 'request_id': 'req-cc052b55-7e7b-4b50-8ef2-f1ac511bc27e'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:23:03.543378 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:23:03.548069 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-cc052b55-7e7b-4b50-8ef2-f1ac511bc27e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5984833842854f5e90646f8b9f32cb63', 'request_id': 'req-cc052b55-7e7b-4b50-8ef2-f1ac511bc27e'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:23:03.560467 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-cc052b55-7e7b-4b50-8ef2-f1ac511bc27e admin admin] The token cache is being invalidate because project 5984833842854f5e90646f8b9f32cb63 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 12:23:03.560754 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-cc052b55-7e7b-4b50-8ef2-f1ac511bc27e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:23:03.561387 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-cc052b55-7e7b-4b50-8ef2-f1ac511bc27e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:23:03.562909 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1392/2784] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Jul 25 12:23:03 2022] DELETE /identity/v3/projects/5984833842854f5e90646f8b9f32cb63 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:23:03.568007 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d243e940-8a66-4ee8-8275-e28e2aeb5c6c None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:23:03.568468 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d243e940-8a66-4ee8-8275-e28e2aeb5c6c None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:23:03.568562 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d243e940-8a66-4ee8-8275-e28e2aeb5c6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:23:03.593605 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-d243e940-8a66-4ee8-8275-e28e2aeb5c6c None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:23:03.594829 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-d243e940-8a66-4ee8-8275-e28e2aeb5c6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:23:03.624903 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:23:03.657983 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1393/2785] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:23:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:23:03.667194 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e6ec85-c181-4492-a2de-61289820d139 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:23:03.667484 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e6ec85-c181-4492-a2de-61289820d139 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:23:03.667794 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e6ec85-c181-4492-a2de-61289820d139 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:23:03.688322 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-45e6ec85-c181-4492-a2de-61289820d139 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:23:03.689549 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-45e6ec85-c181-4492-a2de-61289820d139 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:23:03.698030 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1393/2786] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:23:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:23:03.702148 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d243e940-8a66-4ee8-8275-e28e2aeb5c6c None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:23:03.712866 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d243e940-8a66-4ee8-8275-e28e2aeb5c6c None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:23:03.717427 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256ce99e-dc84-4524-a383-2b4966630202 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:23:03.718083 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256ce99e-dc84-4524-a383-2b4966630202 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:23:03.718373 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256ce99e-dc84-4524-a383-2b4966630202 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:23:03.718655 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256ce99e-dc84-4524-a383-2b4966630202 admin admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:23:03.719386 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256ce99e-dc84-4524-a383-2b4966630202 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:23:03.719733 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256ce99e-dc84-4524-a383-2b4966630202 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:23:03.721358 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256ce99e-dc84-4524-a383-2b4966630202 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:23:03.732759 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1394/2787] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Jul 25 12:23:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:23:03.736844 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e6ec85-c181-4492-a2de-61289820d139 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:23:03.749301 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e6ec85-c181-4492-a2de-61289820d139 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:23:03.754564 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36eb9b77-1fe9-45fb-9a71-f650f15edd65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:23:03.755217 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36eb9b77-1fe9-45fb-9a71-f650f15edd65 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:23:03.755505 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36eb9b77-1fe9-45fb-9a71-f650f15edd65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:23:03.755820 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36eb9b77-1fe9-45fb-9a71-f650f15edd65 admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:23:03.757013 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36eb9b77-1fe9-45fb-9a71-f650f15edd65 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:23:03.758954 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36eb9b77-1fe9-45fb-9a71-f650f15edd65 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:23:03.786443 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1394/2788] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Jul 25 12:23:03 2022] POST /identity/v3/projects => generated 383 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:23:03.790802 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256ce99e-dc84-4524-a383-2b4966630202 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:23:03.801035 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256ce99e-dc84-4524-a383-2b4966630202 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:23:03.855467 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7d95eb-ae19-4c75-b5df-9047872bc189 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:23:03.856157 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7d95eb-ae19-4c75-b5df-9047872bc189 admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:23:03.856445 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7d95eb-ae19-4c75-b5df-9047872bc189 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:23:03.856727 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7d95eb-ae19-4c75-b5df-9047872bc189 admin admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:23:03.857424 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7d95eb-ae19-4c75-b5df-9047872bc189 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:23:03.859317 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7d95eb-ae19-4c75-b5df-9047872bc189 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:23:03.879410 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1395/2789] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Jul 25 12:23:03 2022] POST /identity/v3/users => generated 292 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:23:03.884121 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36eb9b77-1fe9-45fb-9a71-f650f15edd65 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:23:03.894278 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36eb9b77-1fe9-45fb-9a71-f650f15edd65 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:23:03.899942 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c609f2-fe3d-4c63-a6f0-4a35328b18de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:23:03.900601 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c609f2-fe3d-4c63-a6f0-4a35328b18de admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:23:03.900892 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c609f2-fe3d-4c63-a6f0-4a35328b18de admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:23:03.901175 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c609f2-fe3d-4c63-a6f0-4a35328b18de admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:23:03.901906 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c609f2-fe3d-4c63-a6f0-4a35328b18de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:23:03.903502 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c609f2-fe3d-4c63-a6f0-4a35328b18de admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:23:03.913710 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1395/2790] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Jul 25 12:23:03 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:23:03.917473 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7d95eb-ae19-4c75-b5df-9047872bc189 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:23:03.927161 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7d95eb-ae19-4c75-b5df-9047872bc189 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:23:03.931726 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4664507c-91da-40d0-b2fb-a39bb8d2285e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:23:03.932368 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4664507c-91da-40d0-b2fb-a39bb8d2285e admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:23:03.932660 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4664507c-91da-40d0-b2fb-a39bb8d2285e admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:23:03.932938 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4664507c-91da-40d0-b2fb-a39bb8d2285e admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:23:03.933665 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4664507c-91da-40d0-b2fb-a39bb8d2285e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:23:03.935227 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4664507c-91da-40d0-b2fb-a39bb8d2285e admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:23:03.946132 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1396/2791] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Jul 25 12:23:03 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:23:03.950157 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c609f2-fe3d-4c63-a6f0-4a35328b18de admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:23:03.960636 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c609f2-fe3d-4c63-a6f0-4a35328b18de admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:23:03.965225 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351843b9-2c29-432d-9468-baadceac4cbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:23:03.966088 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351843b9-2c29-432d-9468-baadceac4cbb admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:23:03.966375 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351843b9-2c29-432d-9468-baadceac4cbb admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:23:03.966674 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351843b9-2c29-432d-9468-baadceac4cbb admin admin] PATH_INFO: `/v3/projects/2f1e05c3c2464a9d998834a7c76338a2/users/348bd12fa8cd441dab30dc68c5ab9428/roles/472313ae69174578824df0bed6390744` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:23:03.980750 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351843b9-2c29-432d-9468-baadceac4cbb admin admin] RBAC: Authorizing `identity:create_grant(project_id=2f1e05c3c2464a9d998834a7c76338a2, user_id=348bd12fa8cd441dab30dc68c5ab9428, role_id=472313ae69174578824df0bed6390744)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:23:03.983362 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351843b9-2c29-432d-9468-baadceac4cbb admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:23:03.996743 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1396/2792] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Jul 25 12:23:03 2022] PUT /identity/v3/projects/2f1e05c3c2464a9d998834a7c76338a2/users/348bd12fa8cd441dab30dc68c5ab9428/roles/472313ae69174578824df0bed6390744 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:23:04.008043 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11d80b8-a6e0-4f62-ba06-1e2c1eb96e4b None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:23:04.008335 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11d80b8-a6e0-4f62-ba06-1e2c1eb96e4b None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:23:04.008617 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11d80b8-a6e0-4f62-ba06-1e2c1eb96e4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:23:04.035921 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-e11d80b8-a6e0-4f62-ba06-1e2c1eb96e4b None None] MFA Rules not processed for user `348bd12fa8cd441dab30dc68c5ab9428`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:23:04.037120 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-e11d80b8-a6e0-4f62-ba06-1e2c1eb96e4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:23:04.099087 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1397/2793] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:23:04 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:23:04.103466 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351843b9-2c29-432d-9468-baadceac4cbb admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:23:04.162278 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351843b9-2c29-432d-9468-baadceac4cbb admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:23:04.166855 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48373770-cfca-42c7-b8db-fa99e28cedc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:23:04.167503 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48373770-cfca-42c7-b8db-fa99e28cedc2 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:23:04.167815 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48373770-cfca-42c7-b8db-fa99e28cedc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:23:04.168093 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48373770-cfca-42c7-b8db-fa99e28cedc2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:23:04.168772 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48373770-cfca-42c7-b8db-fa99e28cedc2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:23:04.170648 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48373770-cfca-42c7-b8db-fa99e28cedc2 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:23:04.199052 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1397/2794] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Jul 25 12:23:04 2022] POST /identity/v3/projects => generated 381 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:23:04.201685 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11d80b8-a6e0-4f62-ba06-1e2c1eb96e4b None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:23:04.212379 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11d80b8-a6e0-4f62-ba06-1e2c1eb96e4b None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:23:04.242302 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:23:04.273749 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c979f3-8575-48d5-85e2-6b7b8266b864 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:23:04.274426 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c979f3-8575-48d5-85e2-6b7b8266b864 admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:23:04.274716 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c979f3-8575-48d5-85e2-6b7b8266b864 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:23:04.275000 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c979f3-8575-48d5-85e2-6b7b8266b864 admin admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:23:04.275749 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c979f3-8575-48d5-85e2-6b7b8266b864 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:23:04.277667 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c979f3-8575-48d5-85e2-6b7b8266b864 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:23:04.298614 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1398/2795] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Jul 25 12:23:04 2022] POST /identity/v3/users => generated 297 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:23:04.302581 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48373770-cfca-42c7-b8db-fa99e28cedc2 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:23:04.313205 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48373770-cfca-42c7-b8db-fa99e28cedc2 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:23:04.317746 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c54397-9e24-40d6-8909-ffc52ae0fdcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:23:04.318444 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c54397-9e24-40d6-8909-ffc52ae0fdcb admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:23:04.318734 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c54397-9e24-40d6-8909-ffc52ae0fdcb admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:23:04.319625 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c54397-9e24-40d6-8909-ffc52ae0fdcb admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:23:04.320383 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c54397-9e24-40d6-8909-ffc52ae0fdcb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:23:04.321966 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c54397-9e24-40d6-8909-ffc52ae0fdcb admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:23:04.332093 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1398/2796] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Jul 25 12:23:04 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:23:04.335973 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c979f3-8575-48d5-85e2-6b7b8266b864 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:23:04.345956 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c979f3-8575-48d5-85e2-6b7b8266b864 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:23:04.351184 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77be92c-da5c-4b0e-85fc-306ba74227ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:23:04.352090 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77be92c-da5c-4b0e-85fc-306ba74227ab admin admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:23:04.352384 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77be92c-da5c-4b0e-85fc-306ba74227ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:23:04.352668 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77be92c-da5c-4b0e-85fc-306ba74227ab admin admin] PATH_INFO: `/v3/projects/d28b0345b3734fe2b78b445c17f93993/users/a261bb767ed54e6a93e2d0bb266af9da/roles/a7d20885b6d94928a34b3cc5b3a00006` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:23:04.367914 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77be92c-da5c-4b0e-85fc-306ba74227ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=d28b0345b3734fe2b78b445c17f93993, user_id=a261bb767ed54e6a93e2d0bb266af9da, role_id=a7d20885b6d94928a34b3cc5b3a00006)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:23:04.370538 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77be92c-da5c-4b0e-85fc-306ba74227ab admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:23:04.383568 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1399/2797] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Jul 25 12:23:04 2022] PUT /identity/v3/projects/d28b0345b3734fe2b78b445c17f93993/users/a261bb767ed54e6a93e2d0bb266af9da/roles/a7d20885b6d94928a34b3cc5b3a00006 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:23:04.387460 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c54397-9e24-40d6-8909-ffc52ae0fdcb admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:23:04.397548 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c54397-9e24-40d6-8909-ffc52ae0fdcb admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:23:04.447066 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:23:04.454405 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7a288e-1505-4439-ad22-80d21ea56fe5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:23:04.455465 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7a288e-1505-4439-ad22-80d21ea56fe5 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:23:04.455930 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7a288e-1505-4439-ad22-80d21ea56fe5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:23:04.456589 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7a288e-1505-4439-ad22-80d21ea56fe5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:23:04.457972 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7a288e-1505-4439-ad22-80d21ea56fe5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:23:04.459674 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7a288e-1505-4439-ad22-80d21ea56fe5 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:23:04.471709 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1399/2798] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Jul 25 12:23:04 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:23:04.476589 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77be92c-da5c-4b0e-85fc-306ba74227ab admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:23:04.488898 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77be92c-da5c-4b0e-85fc-306ba74227ab admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:23:04.496495 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197e9aa0-bab3-4b79-8cb5-77e1da719c0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:23:04.497473 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197e9aa0-bab3-4b79-8cb5-77e1da719c0f admin admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:23:04.497753 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197e9aa0-bab3-4b79-8cb5-77e1da719c0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:23:04.498037 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197e9aa0-bab3-4b79-8cb5-77e1da719c0f admin admin] PATH_INFO: `/v3/projects/d28b0345b3734fe2b78b445c17f93993/users/a261bb767ed54e6a93e2d0bb266af9da/roles/a7d20885b6d94928a34b3cc5b3a00006` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:23:04.500111 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197e9aa0-bab3-4b79-8cb5-77e1da719c0f admin admin] RBAC: Authorizing `identity:create_grant(project_id=d28b0345b3734fe2b78b445c17f93993, user_id=a261bb767ed54e6a93e2d0bb266af9da, role_id=a7d20885b6d94928a34b3cc5b3a00006)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:23:04.502743 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197e9aa0-bab3-4b79-8cb5-77e1da719c0f admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:23:04.513378 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1400/2799] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Jul 25 12:23:04 2022] PUT /identity/v3/projects/d28b0345b3734fe2b78b445c17f93993/users/a261bb767ed54e6a93e2d0bb266af9da/roles/a7d20885b6d94928a34b3cc5b3a00006 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:23:04.523786 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8091c5ca-e4ef-4230-840b-7849ed4101b5 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:23:04.524081 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8091c5ca-e4ef-4230-840b-7849ed4101b5 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:23:04.524367 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8091c5ca-e4ef-4230-840b-7849ed4101b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:23:04.551797 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-8091c5ca-e4ef-4230-840b-7849ed4101b5 None None] MFA Rules not processed for user `a261bb767ed54e6a93e2d0bb266af9da`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:23:04.552990 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-8091c5ca-e4ef-4230-840b-7849ed4101b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:23:04.618621 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1400/2800] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:23:04 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:23:04.630197 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197e9aa0-bab3-4b79-8cb5-77e1da719c0f admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:23:04.632183 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-197e9aa0-bab3-4b79-8cb5-77e1da719c0f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:23:04.711167 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197e9aa0-bab3-4b79-8cb5-77e1da719c0f admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:23:04.714745 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a59058-fd00-4e14-aa41-ca76aa40403f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f89ce6f0cf04373a414d927fa0f8a7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:23:04.715389 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a59058-fd00-4e14-aa41-ca76aa40403f service cinder] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:23:04.715707 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a59058-fd00-4e14-aa41-ca76aa40403f service cinder] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:23:04.715990 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a59058-fd00-4e14-aa41-ca76aa40403f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:23:04.724947 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a59058-fd00-4e14-aa41-ca76aa40403f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:23:04.727053 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a59058-fd00-4e14-aa41-ca76aa40403f service cinder] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:23:04.736588 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1401/2801] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:23:04 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:23:04.824319 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8091c5ca-e4ef-4230-840b-7849ed4101b5 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:23:04.833806 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8091c5ca-e4ef-4230-840b-7849ed4101b5 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:23:04.837804 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca73882-3b0c-4d02-9827-8c01134341e2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f89ce6f0cf04373a414d927fa0f8a7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:23:04.838436 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca73882-3b0c-4d02-9827-8c01134341e2 service cinder] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:23:04.838718 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca73882-3b0c-4d02-9827-8c01134341e2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:23:04.838994 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca73882-3b0c-4d02-9827-8c01134341e2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:23:04.894196 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca73882-3b0c-4d02-9827-8c01134341e2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:23:04.895923 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca73882-3b0c-4d02-9827-8c01134341e2 service cinder] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:23:04.905182 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1401/2802] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:23:04 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:23:05.161666 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a59058-fd00-4e14-aa41-ca76aa40403f service cinder] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:23:05.222900 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a59058-fd00-4e14-aa41-ca76aa40403f service cinder] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:23:05.227441 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cef785-92ba-4b74-bc76-4cf3bf10add8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:23:05.228332 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cef785-92ba-4b74-bc76-4cf3bf10add8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:23:05.228622 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cef785-92ba-4b74-bc76-4cf3bf10add8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:23:05.228957 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cef785-92ba-4b74-bc76-4cf3bf10add8 admin admin] PATH_INFO: `/v3/users/348bd12fa8cd441dab30dc68c5ab9428` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:23:05.230103 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2cef785-92ba-4b74-bc76-4cf3bf10add8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=348bd12fa8cd441dab30dc68c5ab9428)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:23:05.232244 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2cef785-92ba-4b74-bc76-4cf3bf10add8 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:23:05.237444 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:23:05.278865 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-b2cef785-92ba-4b74-bc76-4cf3bf10add8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '348bd12fa8cd441dab30dc68c5ab9428'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:23:05.279527 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-b2cef785-92ba-4b74-bc76-4cf3bf10add8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '348bd12fa8cd441dab30dc68c5ab9428'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:23:05.286988 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-b2cef785-92ba-4b74-bc76-4cf3bf10add8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '348bd12fa8cd441dab30dc68c5ab9428'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:23:05.299060 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-b2cef785-92ba-4b74-bc76-4cf3bf10add8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '348bd12fa8cd441dab30dc68c5ab9428'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:23:05.306276 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-b2cef785-92ba-4b74-bc76-4cf3bf10add8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '348bd12fa8cd441dab30dc68c5ab9428'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:23:05.312132 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1402/2803] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Jul 25 12:23:05 2022] DELETE /identity/v3/users/348bd12fa8cd441dab30dc68c5ab9428 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:23:05.316732 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7800c20-a9cb-4616-8561-6cc4d98f5f3f None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:23:05.317030 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7800c20-a9cb-4616-8561-6cc4d98f5f3f None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:23:05.317316 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7800c20-a9cb-4616-8561-6cc4d98f5f3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:23:05.336763 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-e7800c20-a9cb-4616-8561-6cc4d98f5f3f None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:23:05.337967 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-e7800c20-a9cb-4616-8561-6cc4d98f5f3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:23:05.394792 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1402/2804] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:23:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:23:05.405208 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cef785-92ba-4b74-bc76-4cf3bf10add8 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:23:05.406610 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-b2cef785-92ba-4b74-bc76-4cf3bf10add8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:23:05.463526 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cef785-92ba-4b74-bc76-4cf3bf10add8 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:23:05.467272 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66512f40-919c-478d-b172-76b542a887a6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51733781201243e8abf9eca39b207750', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:23:05.467951 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66512f40-919c-478d-b172-76b542a887a6 service neutron] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:23:05.468244 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66512f40-919c-478d-b172-76b542a887a6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:23:05.468530 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66512f40-919c-478d-b172-76b542a887a6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:23:05.477682 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66512f40-919c-478d-b172-76b542a887a6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:23:05.479056 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66512f40-919c-478d-b172-76b542a887a6 service neutron] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:23:05.488564 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1403/2805] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:23:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:23:05.764322 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7800c20-a9cb-4616-8561-6cc4d98f5f3f None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:23:05.765801 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-e7800c20-a9cb-4616-8561-6cc4d98f5f3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:23:05.778753 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7800c20-a9cb-4616-8561-6cc4d98f5f3f None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:23:05.784469 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a661a0-4e24-470e-981f-c9e9bbf55c15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:23:05.785531 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a661a0-4e24-470e-981f-c9e9bbf55c15 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:23:05.785822 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a661a0-4e24-470e-981f-c9e9bbf55c15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:23:05.786291 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a661a0-4e24-470e-981f-c9e9bbf55c15 admin admin] PATH_INFO: `/v3/projects/2f1e05c3c2464a9d998834a7c76338a2` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:23:05.787327 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a661a0-4e24-470e-981f-c9e9bbf55c15 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2f1e05c3c2464a9d998834a7c76338a2)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:23:05.789341 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a661a0-4e24-470e-981f-c9e9bbf55c15 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:23:05.802265 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:23:05.837163 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-41a661a0-4e24-470e-981f-c9e9bbf55c15 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2f1e05c3c2464a9d998834a7c76338a2', 'request_id': 'req-41a661a0-4e24-470e-981f-c9e9bbf55c15'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:23:05.843990 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-41a661a0-4e24-470e-981f-c9e9bbf55c15 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2f1e05c3c2464a9d998834a7c76338a2', 'request_id': 'req-41a661a0-4e24-470e-981f-c9e9bbf55c15'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:23:05.849373 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-41a661a0-4e24-470e-981f-c9e9bbf55c15 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2f1e05c3c2464a9d998834a7c76338a2', 'request_id': 'req-41a661a0-4e24-470e-981f-c9e9bbf55c15'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:23:05.860680 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-41a661a0-4e24-470e-981f-c9e9bbf55c15 admin admin] The token cache is being invalidate because project 2f1e05c3c2464a9d998834a7c76338a2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 12:23:05.860999 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-41a661a0-4e24-470e-981f-c9e9bbf55c15 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:23:05.861643 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-41a661a0-4e24-470e-981f-c9e9bbf55c15 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:23:05.863184 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1403/2806] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Jul 25 12:23:05 2022] DELETE /identity/v3/projects/2f1e05c3c2464a9d998834a7c76338a2 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:23:05.867773 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66512f40-919c-478d-b172-76b542a887a6 service neutron] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:23:05.877443 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66512f40-919c-478d-b172-76b542a887a6 service neutron] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:23:05.933547 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e52a126-eeaa-4b49-8143-04c165ab7c82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:23:05.934448 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e52a126-eeaa-4b49-8143-04c165ab7c82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:23:05.934733 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e52a126-eeaa-4b49-8143-04c165ab7c82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:23:05.935014 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e52a126-eeaa-4b49-8143-04c165ab7c82 admin admin] PATH_INFO: `/v3/users/a261bb767ed54e6a93e2d0bb266af9da` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:23:05.936247 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e52a126-eeaa-4b49-8143-04c165ab7c82 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a261bb767ed54e6a93e2d0bb266af9da)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:23:05.938146 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e52a126-eeaa-4b49-8143-04c165ab7c82 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:23:05.969818 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:23:05.984937 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-6e52a126-eeaa-4b49-8143-04c165ab7c82 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a261bb767ed54e6a93e2d0bb266af9da'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:23:05.985585 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-6e52a126-eeaa-4b49-8143-04c165ab7c82 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a261bb767ed54e6a93e2d0bb266af9da'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:23:05.992451 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-6e52a126-eeaa-4b49-8143-04c165ab7c82 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a261bb767ed54e6a93e2d0bb266af9da'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:23:06.004348 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-6e52a126-eeaa-4b49-8143-04c165ab7c82 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a261bb767ed54e6a93e2d0bb266af9da'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:23:06.011118 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-6e52a126-eeaa-4b49-8143-04c165ab7c82 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a261bb767ed54e6a93e2d0bb266af9da'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:23:06.016990 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1404/2807] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Jul 25 12:23:05 2022] DELETE /identity/v3/users/a261bb767ed54e6a93e2d0bb266af9da => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:23:06.284343 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a661a0-4e24-470e-981f-c9e9bbf55c15 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:23:06.285882 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-41a661a0-4e24-470e-981f-c9e9bbf55c15 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:23:06.300356 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a661a0-4e24-470e-981f-c9e9bbf55c15 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:23:06.357816 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423a3223-7b70-4028-9323-128ca3caef1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:23:06.358670 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423a3223-7b70-4028-9323-128ca3caef1b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:23:06.358957 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423a3223-7b70-4028-9323-128ca3caef1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:23:06.359238 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423a3223-7b70-4028-9323-128ca3caef1b admin admin] PATH_INFO: `/v3/projects/d28b0345b3734fe2b78b445c17f93993` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:23:06.360450 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423a3223-7b70-4028-9323-128ca3caef1b admin admin] RBAC: Authorizing `identity:delete_project(project_id=d28b0345b3734fe2b78b445c17f93993)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:23:06.362320 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423a3223-7b70-4028-9323-128ca3caef1b admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:23:06.407782 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-423a3223-7b70-4028-9323-128ca3caef1b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd28b0345b3734fe2b78b445c17f93993', 'request_id': 'req-423a3223-7b70-4028-9323-128ca3caef1b'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:23:06.415820 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-423a3223-7b70-4028-9323-128ca3caef1b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd28b0345b3734fe2b78b445c17f93993', 'request_id': 'req-423a3223-7b70-4028-9323-128ca3caef1b'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:23:06.421282 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-423a3223-7b70-4028-9323-128ca3caef1b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd28b0345b3734fe2b78b445c17f93993', 'request_id': 'req-423a3223-7b70-4028-9323-128ca3caef1b'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:23:06.432034 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-423a3223-7b70-4028-9323-128ca3caef1b admin admin] The token cache is being invalidate because project d28b0345b3734fe2b78b445c17f93993 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 12:23:06.432318 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-423a3223-7b70-4028-9323-128ca3caef1b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:23:06.432950 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-423a3223-7b70-4028-9323-128ca3caef1b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:23:06.434472 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1404/2808] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Jul 25 12:23:06 2022] DELETE /identity/v3/projects/d28b0345b3734fe2b78b445c17f93993 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:23:06.440344 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5f652b-a30e-4f83-9b1d-662cd4b79290 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:23:06.440635 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5f652b-a30e-4f83-9b1d-662cd4b79290 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:23:06.440913 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5f652b-a30e-4f83-9b1d-662cd4b79290 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:23:06.460360 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-ca5f652b-a30e-4f83-9b1d-662cd4b79290 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:23:06.461559 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-ca5f652b-a30e-4f83-9b1d-662cd4b79290 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:23:06.519595 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1405/2809] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:23:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 12:23:06.528146 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d908357-00ea-44d3-97c4-c87ed31d2f08 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:23:06.528385 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d908357-00ea-44d3-97c4-c87ed31d2f08 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:23:06.528672 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d908357-00ea-44d3-97c4-c87ed31d2f08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:23:06.548615 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-9d908357-00ea-44d3-97c4-c87ed31d2f08 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:23:06.549810 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-9d908357-00ea-44d3-97c4-c87ed31d2f08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:23:06.557683 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1405/2810] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:23:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:23:06.562152 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5f652b-a30e-4f83-9b1d-662cd4b79290 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:23:06.572711 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5f652b-a30e-4f83-9b1d-662cd4b79290 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:23:06.580858 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bf3a90-299e-4942-95b4-cb5ce4f91f91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:23:06.581531 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bf3a90-299e-4942-95b4-cb5ce4f91f91 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:23:06.581826 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bf3a90-299e-4942-95b4-cb5ce4f91f91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:23:06.582152 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bf3a90-299e-4942-95b4-cb5ce4f91f91 admin admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:23:06.583132 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42bf3a90-299e-4942-95b4-cb5ce4f91f91 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:23:06.583236 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42bf3a90-299e-4942-95b4-cb5ce4f91f91 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:23:06.585590 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42bf3a90-299e-4942-95b4-cb5ce4f91f91 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:23:06.597474 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1406/2811] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Jul 25 12:23:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:23:06.601975 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d908357-00ea-44d3-97c4-c87ed31d2f08 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:23:06.611121 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d908357-00ea-44d3-97c4-c87ed31d2f08 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:23:06.615782 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fcc2ed-bb15-47c6-b172-c502c2fe360b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:23:06.616432 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fcc2ed-bb15-47c6-b172-c502c2fe360b admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:23:06.616719 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fcc2ed-bb15-47c6-b172-c502c2fe360b admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:23:06.617000 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fcc2ed-bb15-47c6-b172-c502c2fe360b admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:23:06.617694 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fcc2ed-bb15-47c6-b172-c502c2fe360b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:23:06.619583 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fcc2ed-bb15-47c6-b172-c502c2fe360b admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:23:06.646274 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1406/2812] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Jul 25 12:23:06 2022] POST /identity/v3/projects => generated 385 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:23:06.651696 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bf3a90-299e-4942-95b4-cb5ce4f91f91 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:23:06.662791 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bf3a90-299e-4942-95b4-cb5ce4f91f91 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:23:06.720569 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48eccf1-c971-4695-9d30-a3831326ab55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:23:06.721244 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48eccf1-c971-4695-9d30-a3831326ab55 admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:23:06.721531 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48eccf1-c971-4695-9d30-a3831326ab55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:23:06.721811 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48eccf1-c971-4695-9d30-a3831326ab55 admin admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:23:06.722514 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48eccf1-c971-4695-9d30-a3831326ab55 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:23:06.724472 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48eccf1-c971-4695-9d30-a3831326ab55 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:23:06.744667 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1407/2813] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Jul 25 12:23:06 2022] POST /identity/v3/users => generated 293 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:23:06.748801 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fcc2ed-bb15-47c6-b172-c502c2fe360b admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:23:06.754156 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:23:06.760849 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fcc2ed-bb15-47c6-b172-c502c2fe360b admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:23:06.766470 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2939426-d0e0-403c-a93d-a4daffec8e84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:23:06.767125 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2939426-d0e0-403c-a93d-a4daffec8e84 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:23:06.767411 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2939426-d0e0-403c-a93d-a4daffec8e84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:23:06.767725 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2939426-d0e0-403c-a93d-a4daffec8e84 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:23:06.768464 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2939426-d0e0-403c-a93d-a4daffec8e84 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:23:06.770054 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2939426-d0e0-403c-a93d-a4daffec8e84 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:23:06.781128 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1407/2814] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Jul 25 12:23:06 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 12:23:06.785159 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48eccf1-c971-4695-9d30-a3831326ab55 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:23:06.798577 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48eccf1-c971-4695-9d30-a3831326ab55 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:23:06.806582 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c55e1e-69e2-4e2a-9420-3b4dd14b5e49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:23:06.807254 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c55e1e-69e2-4e2a-9420-3b4dd14b5e49 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:23:06.807621 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c55e1e-69e2-4e2a-9420-3b4dd14b5e49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:23:06.807846 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c55e1e-69e2-4e2a-9420-3b4dd14b5e49 admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:23:06.808583 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c55e1e-69e2-4e2a-9420-3b4dd14b5e49 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:23:06.810186 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c55e1e-69e2-4e2a-9420-3b4dd14b5e49 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:23:06.822744 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1408/2815] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Jul 25 12:23:06 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 12:23:06.826915 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2939426-d0e0-403c-a93d-a4daffec8e84 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:23:06.837546 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2939426-d0e0-403c-a93d-a4daffec8e84 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:23:06.842225 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5307f84-ca6a-464f-9e90-62afd0a6ec23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:23:06.843102 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5307f84-ca6a-464f-9e90-62afd0a6ec23 admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:23:06.843390 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5307f84-ca6a-464f-9e90-62afd0a6ec23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:23:06.843704 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5307f84-ca6a-464f-9e90-62afd0a6ec23 admin admin] PATH_INFO: `/v3/projects/a087a4660c574399a24119f4d3cd075d/users/12921c676695483389bb7cb6398451d2/roles/472313ae69174578824df0bed6390744` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:23:06.858339 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5307f84-ca6a-464f-9e90-62afd0a6ec23 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a087a4660c574399a24119f4d3cd075d, user_id=12921c676695483389bb7cb6398451d2, role_id=472313ae69174578824df0bed6390744)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:23:06.861230 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5307f84-ca6a-464f-9e90-62afd0a6ec23 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:23:06.874080 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1408/2816] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Jul 25 12:23:06 2022] PUT /identity/v3/projects/a087a4660c574399a24119f4d3cd075d/users/12921c676695483389bb7cb6398451d2/roles/472313ae69174578824df0bed6390744 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:23:06.884884 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822af40b-2928-4760-b030-63309bb6b7fb None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:23:06.885367 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822af40b-2928-4760-b030-63309bb6b7fb None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:23:06.885468 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822af40b-2928-4760-b030-63309bb6b7fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:23:06.910654 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:23:06.914217 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-822af40b-2928-4760-b030-63309bb6b7fb None None] MFA Rules not processed for user `12921c676695483389bb7cb6398451d2`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:23:06.916329 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-822af40b-2928-4760-b030-63309bb6b7fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:23:06.979299 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1409/2817] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:23:06 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:23:06.989842 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5307f84-ca6a-464f-9e90-62afd0a6ec23 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:23:06.992004 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-a5307f84-ca6a-464f-9e90-62afd0a6ec23 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:23:07.047176 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5307f84-ca6a-464f-9e90-62afd0a6ec23 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:23:07.050797 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6266e19-abe8-4011-811e-f1cce62e1e1a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f89ce6f0cf04373a414d927fa0f8a7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:23:07.051433 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6266e19-abe8-4011-811e-f1cce62e1e1a service cinder] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:23:07.051771 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6266e19-abe8-4011-811e-f1cce62e1e1a service cinder] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:23:07.052040 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6266e19-abe8-4011-811e-f1cce62e1e1a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:23:07.060710 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6266e19-abe8-4011-811e-f1cce62e1e1a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:23:07.062129 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6266e19-abe8-4011-811e-f1cce62e1e1a service cinder] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:23:07.071335 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1409/2818] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:23:06 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:23:14.444281 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822af40b-2928-4760-b030-63309bb6b7fb None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:23:14.518770 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822af40b-2928-4760-b030-63309bb6b7fb None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:23:14.523794 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35149041-7276-4cb2-a78a-d3563453e7cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:23:14.524679 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35149041-7276-4cb2-a78a-d3563453e7cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:23:14.524967 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35149041-7276-4cb2-a78a-d3563453e7cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:23:14.525248 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35149041-7276-4cb2-a78a-d3563453e7cb admin admin] PATH_INFO: `/v3/users/12921c676695483389bb7cb6398451d2` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:23:14.526471 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35149041-7276-4cb2-a78a-d3563453e7cb admin admin] RBAC: Authorizing `identity:delete_user(user_id=12921c676695483389bb7cb6398451d2)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:23:14.528434 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35149041-7276-4cb2-a78a-d3563453e7cb admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:23:14.575871 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-35149041-7276-4cb2-a78a-d3563453e7cb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '12921c676695483389bb7cb6398451d2'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:23:14.576543 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-35149041-7276-4cb2-a78a-d3563453e7cb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '12921c676695483389bb7cb6398451d2'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:23:14.583822 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-35149041-7276-4cb2-a78a-d3563453e7cb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '12921c676695483389bb7cb6398451d2'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:23:14.595647 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-35149041-7276-4cb2-a78a-d3563453e7cb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '12921c676695483389bb7cb6398451d2'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:23:14.603103 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-35149041-7276-4cb2-a78a-d3563453e7cb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '12921c676695483389bb7cb6398451d2'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:23:14.609001 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1410/2819] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Jul 25 12:23:14 2022] DELETE /identity/v3/users/12921c676695483389bb7cb6398451d2 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:23:14.613630 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97185892-08b3-41e2-9a57-81a0e9c299bf None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:23:14.614156 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97185892-08b3-41e2-9a57-81a0e9c299bf None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:23:14.614317 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97185892-08b3-41e2-9a57-81a0e9c299bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:23:14.641280 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-97185892-08b3-41e2-9a57-81a0e9c299bf None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:23:14.642507 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-97185892-08b3-41e2-9a57-81a0e9c299bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:23:14.701566 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1410/2820] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:23:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:23:14.711091 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35149041-7276-4cb2-a78a-d3563453e7cb admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:23:14.712539 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-35149041-7276-4cb2-a78a-d3563453e7cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:23:14.739257 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:23:14.774289 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35149041-7276-4cb2-a78a-d3563453e7cb admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:23:14.778828 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06123d7d-8594-452e-8da5-e868568c28d6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51733781201243e8abf9eca39b207750', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:23:14.779494 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06123d7d-8594-452e-8da5-e868568c28d6 service neutron] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:23:14.779820 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06123d7d-8594-452e-8da5-e868568c28d6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:23:14.780106 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06123d7d-8594-452e-8da5-e868568c28d6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:23:14.790175 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06123d7d-8594-452e-8da5-e868568c28d6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:23:14.791644 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06123d7d-8594-452e-8da5-e868568c28d6 service neutron] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:23:14.808040 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1411/2821] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:23:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:23:15.080418 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97185892-08b3-41e2-9a57-81a0e9c299bf None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:23:15.081848 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-97185892-08b3-41e2-9a57-81a0e9c299bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:23:15.093974 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97185892-08b3-41e2-9a57-81a0e9c299bf None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:23:15.099816 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f379c01-bfaa-43bc-8251-7f0d40428ab0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:23:15.100540 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f379c01-bfaa-43bc-8251-7f0d40428ab0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:23:15.101020 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f379c01-bfaa-43bc-8251-7f0d40428ab0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:23:15.101374 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f379c01-bfaa-43bc-8251-7f0d40428ab0 admin admin] PATH_INFO: `/v3/projects/a087a4660c574399a24119f4d3cd075d` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:23:15.102849 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f379c01-bfaa-43bc-8251-7f0d40428ab0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a087a4660c574399a24119f4d3cd075d)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:23:15.104897 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f379c01-bfaa-43bc-8251-7f0d40428ab0 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:23:15.163466 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-6f379c01-bfaa-43bc-8251-7f0d40428ab0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a087a4660c574399a24119f4d3cd075d', 'request_id': 'req-6f379c01-bfaa-43bc-8251-7f0d40428ab0'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:23:15.170982 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-6f379c01-bfaa-43bc-8251-7f0d40428ab0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a087a4660c574399a24119f4d3cd075d', 'request_id': 'req-6f379c01-bfaa-43bc-8251-7f0d40428ab0'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:23:15.176974 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-6f379c01-bfaa-43bc-8251-7f0d40428ab0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a087a4660c574399a24119f4d3cd075d', 'request_id': 'req-6f379c01-bfaa-43bc-8251-7f0d40428ab0'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:23:15.178536 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:23:15.194267 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-6f379c01-bfaa-43bc-8251-7f0d40428ab0 admin admin] The token cache is being invalidate because project a087a4660c574399a24119f4d3cd075d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 12:23:15.194608 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-6f379c01-bfaa-43bc-8251-7f0d40428ab0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:23:15.195477 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-6f379c01-bfaa-43bc-8251-7f0d40428ab0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:23:15.197107 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1411/2822] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Jul 25 12:23:15 2022] DELETE /identity/v3/projects/a087a4660c574399a24119f4d3cd075d => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:23:15.202018 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7085b733-102c-45fe-bcef-5ff52f90963d None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:23:15.202254 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7085b733-102c-45fe-bcef-5ff52f90963d None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:23:15.202543 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7085b733-102c-45fe-bcef-5ff52f90963d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:23:15.222108 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-7085b733-102c-45fe-bcef-5ff52f90963d None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:23:15.223334 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-7085b733-102c-45fe-bcef-5ff52f90963d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:23:15.284879 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1412/2823] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:23:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:23:15.350176 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc409b6-2dd2-4239-9f30-5fd0afbf0274 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:23:15.350472 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc409b6-2dd2-4239-9f30-5fd0afbf0274 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:23:15.350759 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc409b6-2dd2-4239-9f30-5fd0afbf0274 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:23:15.370727 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-9fc409b6-2dd2-4239-9f30-5fd0afbf0274 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:23:15.371981 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-9fc409b6-2dd2-4239-9f30-5fd0afbf0274 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:23:15.380046 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1412/2824] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:23:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:23:15.384306 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7085b733-102c-45fe-bcef-5ff52f90963d None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:23:15.395783 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7085b733-102c-45fe-bcef-5ff52f90963d None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:23:15.400509 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542f22e7-4845-4f2f-b191-de9a0c004dbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:23:15.401174 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542f22e7-4845-4f2f-b191-de9a0c004dbb admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:23:15.401466 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542f22e7-4845-4f2f-b191-de9a0c004dbb admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:23:15.401752 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542f22e7-4845-4f2f-b191-de9a0c004dbb admin admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:23:15.402488 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542f22e7-4845-4f2f-b191-de9a0c004dbb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:23:15.402832 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542f22e7-4845-4f2f-b191-de9a0c004dbb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:23:15.404483 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542f22e7-4845-4f2f-b191-de9a0c004dbb admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:23:15.417110 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1413/2825] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Jul 25 12:23:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:23:15.421538 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc409b6-2dd2-4239-9f30-5fd0afbf0274 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:23:15.433066 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc409b6-2dd2-4239-9f30-5fd0afbf0274 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:23:15.437765 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a93318-8a37-4746-bdec-fb1f7fc5000c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:23:15.438419 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a93318-8a37-4746-bdec-fb1f7fc5000c admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:23:15.438707 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a93318-8a37-4746-bdec-fb1f7fc5000c admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:23:15.438989 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a93318-8a37-4746-bdec-fb1f7fc5000c admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:23:15.439727 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a93318-8a37-4746-bdec-fb1f7fc5000c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:23:15.441641 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a93318-8a37-4746-bdec-fb1f7fc5000c admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:23:15.469988 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1413/2826] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Jul 25 12:23:15 2022] POST /identity/v3/projects => generated 377 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:23:15.474297 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542f22e7-4845-4f2f-b191-de9a0c004dbb admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:23:15.484986 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542f22e7-4845-4f2f-b191-de9a0c004dbb admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:23:15.542734 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56ea731-c815-4643-b4e7-a6bd54154753 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:23:15.543414 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56ea731-c815-4643-b4e7-a6bd54154753 admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:23:15.543740 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56ea731-c815-4643-b4e7-a6bd54154753 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:23:15.544027 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56ea731-c815-4643-b4e7-a6bd54154753 admin admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:23:15.544744 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56ea731-c815-4643-b4e7-a6bd54154753 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:23:15.546663 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56ea731-c815-4643-b4e7-a6bd54154753 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:23:15.567491 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1414/2827] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Jul 25 12:23:15 2022] POST /identity/v3/users => generated 289 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:23:15.571769 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a93318-8a37-4746-bdec-fb1f7fc5000c admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:23:15.582083 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a93318-8a37-4746-bdec-fb1f7fc5000c admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:23:15.586699 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03ece87-cb64-4f66-b2ae-6b876ef12499 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:23:15.587352 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03ece87-cb64-4f66-b2ae-6b876ef12499 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:23:15.587681 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03ece87-cb64-4f66-b2ae-6b876ef12499 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:23:15.587965 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03ece87-cb64-4f66-b2ae-6b876ef12499 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:23:15.588692 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03ece87-cb64-4f66-b2ae-6b876ef12499 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:23:15.590279 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03ece87-cb64-4f66-b2ae-6b876ef12499 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:23:15.600831 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1414/2828] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Jul 25 12:23:15 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:23:15.605033 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56ea731-c815-4643-b4e7-a6bd54154753 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:23:15.616166 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56ea731-c815-4643-b4e7-a6bd54154753 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:23:15.620890 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f905a42f-f0bf-473c-9a5c-6e20feb499a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:23:15.621589 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f905a42f-f0bf-473c-9a5c-6e20feb499a7 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:23:15.621880 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f905a42f-f0bf-473c-9a5c-6e20feb499a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:23:15.622165 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f905a42f-f0bf-473c-9a5c-6e20feb499a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:23:15.622894 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f905a42f-f0bf-473c-9a5c-6e20feb499a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:23:15.624528 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f905a42f-f0bf-473c-9a5c-6e20feb499a7 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:23:15.635114 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1415/2829] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Jul 25 12:23:15 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:23:15.639194 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03ece87-cb64-4f66-b2ae-6b876ef12499 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:23:15.650242 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03ece87-cb64-4f66-b2ae-6b876ef12499 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:23:15.654802 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1eb8c8-faff-492a-94a1-f3eb0ac6fc74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:23:15.655713 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1eb8c8-faff-492a-94a1-f3eb0ac6fc74 admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:23:15.656004 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1eb8c8-faff-492a-94a1-f3eb0ac6fc74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:23:15.656319 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1eb8c8-faff-492a-94a1-f3eb0ac6fc74 admin admin] PATH_INFO: `/v3/projects/6b94c681189c4496930a4814e971dbd8/users/cb804282c49b4f4894ff4009ef573b2a/roles/472313ae69174578824df0bed6390744` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:23:15.673169 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1eb8c8-faff-492a-94a1-f3eb0ac6fc74 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6b94c681189c4496930a4814e971dbd8, user_id=cb804282c49b4f4894ff4009ef573b2a, role_id=472313ae69174578824df0bed6390744)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:23:15.676196 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1eb8c8-faff-492a-94a1-f3eb0ac6fc74 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:23:15.689672 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1415/2830] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Jul 25 12:23:15 2022] PUT /identity/v3/projects/6b94c681189c4496930a4814e971dbd8/users/cb804282c49b4f4894ff4009ef573b2a/roles/472313ae69174578824df0bed6390744 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:23:15.699649 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e75feb-9f44-4b83-9e00-06a6dd4fd9e6 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:23:15.699943 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e75feb-9f44-4b83-9e00-06a6dd4fd9e6 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:23:15.700221 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e75feb-9f44-4b83-9e00-06a6dd4fd9e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:23:15.728578 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-67e75feb-9f44-4b83-9e00-06a6dd4fd9e6 None None] MFA Rules not processed for user `cb804282c49b4f4894ff4009ef573b2a`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:23:15.729708 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-67e75feb-9f44-4b83-9e00-06a6dd4fd9e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:23:15.748290 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:23:15.803705 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1416/2831] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:23:15 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:23:15.814868 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1eb8c8-faff-492a-94a1-f3eb0ac6fc74 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:23:15.816589 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-9f1eb8c8-faff-492a-94a1-f3eb0ac6fc74 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:23:15.875787 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1eb8c8-faff-492a-94a1-f3eb0ac6fc74 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:23:15.879365 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1908ffe5-917a-4f8c-b79a-6a9a8960f935 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f89ce6f0cf04373a414d927fa0f8a7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:23:15.880032 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1908ffe5-917a-4f8c-b79a-6a9a8960f935 service cinder] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:23:15.880316 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1908ffe5-917a-4f8c-b79a-6a9a8960f935 service cinder] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:23:15.880595 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1908ffe5-917a-4f8c-b79a-6a9a8960f935 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:23:15.888671 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1908ffe5-917a-4f8c-b79a-6a9a8960f935 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:23:15.890091 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1908ffe5-917a-4f8c-b79a-6a9a8960f935 service cinder] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:23:15.899401 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1416/2832] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:23:15 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:23:25.800723 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=18 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:23:25.894968 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=56,keystone:INSERT=2 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:23:36.559991 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e75feb-9f44-4b83-9e00-06a6dd4fd9e6 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:23:36.622657 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e75feb-9f44-4b83-9e00-06a6dd4fd9e6 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:23:36.627200 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec25d0e6-07de-4ef1-b29b-d17d9e913952 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:23:36.628096 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec25d0e6-07de-4ef1-b29b-d17d9e913952 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:23:36.628390 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec25d0e6-07de-4ef1-b29b-d17d9e913952 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:23:36.628676 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec25d0e6-07de-4ef1-b29b-d17d9e913952 admin admin] PATH_INFO: `/v3/users/cb804282c49b4f4894ff4009ef573b2a` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:23:36.629877 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec25d0e6-07de-4ef1-b29b-d17d9e913952 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cb804282c49b4f4894ff4009ef573b2a)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:23:36.631824 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec25d0e6-07de-4ef1-b29b-d17d9e913952 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:23:36.678100 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-ec25d0e6-07de-4ef1-b29b-d17d9e913952 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cb804282c49b4f4894ff4009ef573b2a'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:23:36.678756 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-ec25d0e6-07de-4ef1-b29b-d17d9e913952 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cb804282c49b4f4894ff4009ef573b2a'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:23:36.686503 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-ec25d0e6-07de-4ef1-b29b-d17d9e913952 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cb804282c49b4f4894ff4009ef573b2a'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:23:36.699080 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-ec25d0e6-07de-4ef1-b29b-d17d9e913952 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cb804282c49b4f4894ff4009ef573b2a'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:23:36.705836 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-ec25d0e6-07de-4ef1-b29b-d17d9e913952 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cb804282c49b4f4894ff4009ef573b2a'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:23:36.711723 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1417/2833] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Jul 25 12:23:36 2022] DELETE /identity/v3/users/cb804282c49b4f4894ff4009ef573b2a => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:23:36.716170 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28118004-9d43-4cc2-b1b9-f13a94162af7 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:23:36.716666 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28118004-9d43-4cc2-b1b9-f13a94162af7 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:23:36.716776 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28118004-9d43-4cc2-b1b9-f13a94162af7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:23:36.734527 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-28118004-9d43-4cc2-b1b9-f13a94162af7 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:23:36.735761 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-28118004-9d43-4cc2-b1b9-f13a94162af7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:23:36.795644 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1417/2834] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:23:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:23:36.805429 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec25d0e6-07de-4ef1-b29b-d17d9e913952 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:23:36.807636 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-ec25d0e6-07de-4ef1-b29b-d17d9e913952 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:23:36.868947 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec25d0e6-07de-4ef1-b29b-d17d9e913952 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:23:36.873344 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bc3ccb-cd74-45d1-af4e-0c43a4bc0627 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51733781201243e8abf9eca39b207750', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:23:36.874028 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bc3ccb-cd74-45d1-af4e-0c43a4bc0627 service neutron] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:23:36.874318 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bc3ccb-cd74-45d1-af4e-0c43a4bc0627 service neutron] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:23:36.874601 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bc3ccb-cd74-45d1-af4e-0c43a4bc0627 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:23:36.884663 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bc3ccb-cd74-45d1-af4e-0c43a4bc0627 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:23:36.886806 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bc3ccb-cd74-45d1-af4e-0c43a4bc0627 service neutron] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:23:36.897448 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1418/2835] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:23:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:23:37.172525 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28118004-9d43-4cc2-b1b9-f13a94162af7 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:23:37.174039 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-28118004-9d43-4cc2-b1b9-f13a94162af7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:23:37.190452 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28118004-9d43-4cc2-b1b9-f13a94162af7 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:23:37.195052 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c0e876-2917-4fb6-b46a-e2561048b3ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:23:37.195943 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c0e876-2917-4fb6-b46a-e2561048b3ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:23:37.196221 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c0e876-2917-4fb6-b46a-e2561048b3ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:23:37.197078 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c0e876-2917-4fb6-b46a-e2561048b3ac admin admin] PATH_INFO: `/v3/projects/6b94c681189c4496930a4814e971dbd8` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:23:37.198976 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c0e876-2917-4fb6-b46a-e2561048b3ac admin admin] RBAC: Authorizing `identity:delete_project(project_id=6b94c681189c4496930a4814e971dbd8)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:23:37.200921 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c0e876-2917-4fb6-b46a-e2561048b3ac admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:23:37.246908 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-01c0e876-2917-4fb6-b46a-e2561048b3ac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6b94c681189c4496930a4814e971dbd8', 'request_id': 'req-01c0e876-2917-4fb6-b46a-e2561048b3ac'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:23:37.254269 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-01c0e876-2917-4fb6-b46a-e2561048b3ac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6b94c681189c4496930a4814e971dbd8', 'request_id': 'req-01c0e876-2917-4fb6-b46a-e2561048b3ac'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:23:37.260611 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-01c0e876-2917-4fb6-b46a-e2561048b3ac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6b94c681189c4496930a4814e971dbd8', 'request_id': 'req-01c0e876-2917-4fb6-b46a-e2561048b3ac'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:23:37.271747 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-01c0e876-2917-4fb6-b46a-e2561048b3ac admin admin] The token cache is being invalidate because project 6b94c681189c4496930a4814e971dbd8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 12:23:37.272041 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-01c0e876-2917-4fb6-b46a-e2561048b3ac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:23:37.272707 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-01c0e876-2917-4fb6-b46a-e2561048b3ac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:23:37.274235 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1418/2836] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Jul 25 12:23:37 2022] DELETE /identity/v3/projects/6b94c681189c4496930a4814e971dbd8 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:23:37.279268 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c05561-ade7-4fcf-8596-4f2c780aa63d None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:23:37.279592 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c05561-ade7-4fcf-8596-4f2c780aa63d None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:23:37.279882 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c05561-ade7-4fcf-8596-4f2c780aa63d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:23:37.299102 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-22c05561-ade7-4fcf-8596-4f2c780aa63d None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:23:37.300406 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-22c05561-ade7-4fcf-8596-4f2c780aa63d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:23:37.317781 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:23:37.361819 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1419/2837] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:23:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:23:37.370053 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b636b5-87e3-4041-b62d-4b431b95b579 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:23:37.370365 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b636b5-87e3-4041-b62d-4b431b95b579 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:23:37.370645 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b636b5-87e3-4041-b62d-4b431b95b579 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:23:37.390217 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-45b636b5-87e3-4041-b62d-4b431b95b579 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:23:37.391418 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-45b636b5-87e3-4041-b62d-4b431b95b579 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:23:37.399507 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1419/2838] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:23:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:23:37.403486 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c05561-ade7-4fcf-8596-4f2c780aa63d None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:23:37.413026 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c05561-ade7-4fcf-8596-4f2c780aa63d None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:23:37.417669 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a41b6c4-40b9-4593-8421-e18f38cea405 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:23:37.418321 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a41b6c4-40b9-4593-8421-e18f38cea405 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:23:37.418611 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a41b6c4-40b9-4593-8421-e18f38cea405 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:23:37.418893 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a41b6c4-40b9-4593-8421-e18f38cea405 admin admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:23:37.419662 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a41b6c4-40b9-4593-8421-e18f38cea405 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:23:37.419978 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a41b6c4-40b9-4593-8421-e18f38cea405 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:23:37.421565 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a41b6c4-40b9-4593-8421-e18f38cea405 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:23:37.433464 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1420/2839] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Jul 25 12:23:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:23:37.437580 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b636b5-87e3-4041-b62d-4b431b95b579 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:23:37.446747 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b636b5-87e3-4041-b62d-4b431b95b579 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:23:37.451922 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79009e7-e0bf-4540-b8bd-5916dbd2f241 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:23:37.452564 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79009e7-e0bf-4540-b8bd-5916dbd2f241 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:23:37.452853 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79009e7-e0bf-4540-b8bd-5916dbd2f241 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:23:37.453130 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79009e7-e0bf-4540-b8bd-5916dbd2f241 admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:23:37.453816 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79009e7-e0bf-4540-b8bd-5916dbd2f241 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:23:37.455733 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79009e7-e0bf-4540-b8bd-5916dbd2f241 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:23:37.482831 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1420/2840] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Jul 25 12:23:37 2022] POST /identity/v3/projects => generated 381 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:23:37.486915 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a41b6c4-40b9-4593-8421-e18f38cea405 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:23:37.498311 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a41b6c4-40b9-4593-8421-e18f38cea405 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:23:37.558454 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9868df-5c21-472a-903b-ab6a3691e2be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:23:37.559135 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9868df-5c21-472a-903b-ab6a3691e2be admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:23:37.559427 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9868df-5c21-472a-903b-ab6a3691e2be admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:23:37.559749 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9868df-5c21-472a-903b-ab6a3691e2be admin admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:23:37.560462 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9868df-5c21-472a-903b-ab6a3691e2be admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:23:37.562396 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9868df-5c21-472a-903b-ab6a3691e2be admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:23:37.583477 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1421/2841] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Jul 25 12:23:37 2022] POST /identity/v3/users => generated 291 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:23:37.587523 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79009e7-e0bf-4540-b8bd-5916dbd2f241 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:23:37.597722 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79009e7-e0bf-4540-b8bd-5916dbd2f241 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:23:37.602318 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bdc675-706f-4865-9bd3-fa51a13af7cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:23:37.602975 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bdc675-706f-4865-9bd3-fa51a13af7cf admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:23:37.603265 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bdc675-706f-4865-9bd3-fa51a13af7cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:23:37.604111 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bdc675-706f-4865-9bd3-fa51a13af7cf admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:23:37.604848 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8bdc675-706f-4865-9bd3-fa51a13af7cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:23:37.606450 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8bdc675-706f-4865-9bd3-fa51a13af7cf admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:23:37.616185 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1421/2842] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Jul 25 12:23:37 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 12:23:37.620546 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9868df-5c21-472a-903b-ab6a3691e2be admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:23:37.630490 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9868df-5c21-472a-903b-ab6a3691e2be admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:23:37.635192 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ebb118-541d-44c3-8332-f0f5538c9ffc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:23:37.635912 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ebb118-541d-44c3-8332-f0f5538c9ffc admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:23:37.636189 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ebb118-541d-44c3-8332-f0f5538c9ffc admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:23:37.636475 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ebb118-541d-44c3-8332-f0f5538c9ffc admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:23:37.637200 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ebb118-541d-44c3-8332-f0f5538c9ffc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:23:37.639083 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ebb118-541d-44c3-8332-f0f5538c9ffc admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:23:37.650974 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1422/2843] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Jul 25 12:23:37 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:23:37.655057 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bdc675-706f-4865-9bd3-fa51a13af7cf admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:23:37.665048 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bdc675-706f-4865-9bd3-fa51a13af7cf admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:23:37.670202 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebb445a-c70d-4782-ab25-7ec375d866b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:23:37.671073 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebb445a-c70d-4782-ab25-7ec375d866b0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:23:37.671360 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebb445a-c70d-4782-ab25-7ec375d866b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:23:37.671820 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebb445a-c70d-4782-ab25-7ec375d866b0 admin admin] PATH_INFO: `/v3/projects/a978671fc17b439eb73537f087a72e53/users/4adc3e648c6d4bd9830dc064b9bc4e70/roles/472313ae69174578824df0bed6390744` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:23:37.686765 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebb445a-c70d-4782-ab25-7ec375d866b0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a978671fc17b439eb73537f087a72e53, user_id=4adc3e648c6d4bd9830dc064b9bc4e70, role_id=472313ae69174578824df0bed6390744)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:23:37.689430 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebb445a-c70d-4782-ab25-7ec375d866b0 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:23:37.702567 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1422/2844] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Jul 25 12:23:37 2022] PUT /identity/v3/projects/a978671fc17b439eb73537f087a72e53/users/4adc3e648c6d4bd9830dc064b9bc4e70/roles/472313ae69174578824df0bed6390744 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:23:37.707575 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658e5fc3-13ac-434b-80af-032d3233cf4d None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:23:37.707874 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658e5fc3-13ac-434b-80af-032d3233cf4d None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:23:37.708164 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658e5fc3-13ac-434b-80af-032d3233cf4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:23:37.726828 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-658e5fc3-13ac-434b-80af-032d3233cf4d None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:23:37.728072 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-658e5fc3-13ac-434b-80af-032d3233cf4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:23:37.787512 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1423/2845] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:23:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 12:23:37.799257 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebb445a-c70d-4782-ab25-7ec375d866b0 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:23:37.801130 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-bebb445a-c70d-4782-ab25-7ec375d866b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:23:37.823199 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:23:37.862556 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebb445a-c70d-4782-ab25-7ec375d866b0 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:23:37.867915 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289042c9-22d9-413b-9029-87bb43789e3c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51733781201243e8abf9eca39b207750', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:23:37.868783 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289042c9-22d9-413b-9029-87bb43789e3c service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:23:37.869103 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289042c9-22d9-413b-9029-87bb43789e3c service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:23:37.869929 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289042c9-22d9-413b-9029-87bb43789e3c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:23:37.879656 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289042c9-22d9-413b-9029-87bb43789e3c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:23:37.881103 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289042c9-22d9-413b-9029-87bb43789e3c service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:23:37.890589 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1423/2846] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:23:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:23:37.951974 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658e5fc3-13ac-434b-80af-032d3233cf4d None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:23:37.953578 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-658e5fc3-13ac-434b-80af-032d3233cf4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:23:37.965916 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658e5fc3-13ac-434b-80af-032d3233cf4d None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:23:37.971637 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e7e0a4-0520-4f94-b1e2-e7625ac19187 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:23:37.972605 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e7e0a4-0520-4f94-b1e2-e7625ac19187 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:23:37.972915 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e7e0a4-0520-4f94-b1e2-e7625ac19187 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:23:37.973198 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e7e0a4-0520-4f94-b1e2-e7625ac19187 admin admin] PATH_INFO: `/v3/users/4c754338e83f4f83931be7c576015cd4` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:23:37.974517 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e7e0a4-0520-4f94-b1e2-e7625ac19187 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4c754338e83f4f83931be7c576015cd4)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:23:37.976718 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e7e0a4-0520-4f94-b1e2-e7625ac19187 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:23:38.003492 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=93,keystone:INSERT=3,keystone:DELETE=4 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:23:38.030913 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-53e7e0a4-0520-4f94-b1e2-e7625ac19187 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4c754338e83f4f83931be7c576015cd4'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:23:38.031678 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-53e7e0a4-0520-4f94-b1e2-e7625ac19187 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4c754338e83f4f83931be7c576015cd4'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:23:38.039726 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-53e7e0a4-0520-4f94-b1e2-e7625ac19187 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4c754338e83f4f83931be7c576015cd4'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:23:38.052820 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-53e7e0a4-0520-4f94-b1e2-e7625ac19187 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4c754338e83f4f83931be7c576015cd4'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:23:38.064509 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-53e7e0a4-0520-4f94-b1e2-e7625ac19187 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4c754338e83f4f83931be7c576015cd4'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:23:38.071527 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1424/2847] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Jul 25 12:23:37 2022] DELETE /identity/v3/users/4c754338e83f4f83931be7c576015cd4 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:23:38.076124 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f614bf-bd0b-406c-b991-639ed503c541 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:23:38.076420 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f614bf-bd0b-406c-b991-639ed503c541 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:23:38.076707 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f614bf-bd0b-406c-b991-639ed503c541 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:23:38.096349 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-67f614bf-bd0b-406c-b991-639ed503c541 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:23:38.097577 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-67f614bf-bd0b-406c-b991-639ed503c541 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:23:38.163306 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1424/2848] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:23:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:23:38.173211 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e7e0a4-0520-4f94-b1e2-e7625ac19187 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:23:38.174605 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-53e7e0a4-0520-4f94-b1e2-e7625ac19187 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:23:38.242904 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e7e0a4-0520-4f94-b1e2-e7625ac19187 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:23:38.248292 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df0dbf7-ff65-4914-a1cd-da92573f1f7c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51733781201243e8abf9eca39b207750', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:23:38.249081 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df0dbf7-ff65-4914-a1cd-da92573f1f7c service neutron] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:23:38.249385 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df0dbf7-ff65-4914-a1cd-da92573f1f7c service neutron] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:23:38.249672 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df0dbf7-ff65-4914-a1cd-da92573f1f7c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:23:38.261051 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df0dbf7-ff65-4914-a1cd-da92573f1f7c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:23:38.262571 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df0dbf7-ff65-4914-a1cd-da92573f1f7c service neutron] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:23:38.276397 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1425/2849] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:23:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:23:38.644685 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa1fae5-a00f-4b41-83c6-04c93a30ba57 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:23:38.645138 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa1fae5-a00f-4b41-83c6-04c93a30ba57 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:23:38.645433 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa1fae5-a00f-4b41-83c6-04c93a30ba57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:23:38.677101 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-9fa1fae5-a00f-4b41-83c6-04c93a30ba57 None None] MFA Rules not processed for user `4adc3e648c6d4bd9830dc064b9bc4e70`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:23:38.678759 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-9fa1fae5-a00f-4b41-83c6-04c93a30ba57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:23:38.689500 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df0dbf7-ff65-4914-a1cd-da92573f1f7c service neutron] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:23:38.691028 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-5df0dbf7-ff65-4914-a1cd-da92573f1f7c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:23:38.714678 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df0dbf7-ff65-4914-a1cd-da92573f1f7c service neutron] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:23:38.719759 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43461bb0-2071-433c-9ea2-29e7b2cc0a22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:23:38.720489 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43461bb0-2071-433c-9ea2-29e7b2cc0a22 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:23:38.720906 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43461bb0-2071-433c-9ea2-29e7b2cc0a22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:23:38.721444 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43461bb0-2071-433c-9ea2-29e7b2cc0a22 admin admin] PATH_INFO: `/v3/projects/ccb7de30a6914b489252b2f2ce54f1b9` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:23:38.722992 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43461bb0-2071-433c-9ea2-29e7b2cc0a22 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ccb7de30a6914b489252b2f2ce54f1b9)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:23:38.725227 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43461bb0-2071-433c-9ea2-29e7b2cc0a22 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:23:38.780360 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-43461bb0-2071-433c-9ea2-29e7b2cc0a22 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ccb7de30a6914b489252b2f2ce54f1b9', 'request_id': 'req-43461bb0-2071-433c-9ea2-29e7b2cc0a22'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:23:38.783072 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1425/2850] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:23:38 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:23:38.791614 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-43461bb0-2071-433c-9ea2-29e7b2cc0a22 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ccb7de30a6914b489252b2f2ce54f1b9', 'request_id': 'req-43461bb0-2071-433c-9ea2-29e7b2cc0a22'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:23:38.795519 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa1fae5-a00f-4b41-83c6-04c93a30ba57 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:23:38.797517 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-9fa1fae5-a00f-4b41-83c6-04c93a30ba57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:23:38.800064 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-43461bb0-2071-433c-9ea2-29e7b2cc0a22 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ccb7de30a6914b489252b2f2ce54f1b9', 'request_id': 'req-43461bb0-2071-433c-9ea2-29e7b2cc0a22'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:23:38.812086 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-43461bb0-2071-433c-9ea2-29e7b2cc0a22 admin admin] The token cache is being invalidate because project ccb7de30a6914b489252b2f2ce54f1b9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80298) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 12:23:38.812385 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-43461bb0-2071-433c-9ea2-29e7b2cc0a22 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:23:38.813130 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-43461bb0-2071-433c-9ea2-29e7b2cc0a22 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:23:38.814931 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1426/2851] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Jul 25 12:23:38 2022] DELETE /identity/v3/projects/ccb7de30a6914b489252b2f2ce54f1b9 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:23:38.866023 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa1fae5-a00f-4b41-83c6-04c93a30ba57 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:23:38.871237 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:23:38.922302 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d78bd3d-1d20-4150-8fe1-6e88412a0d5f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f89ce6f0cf04373a414d927fa0f8a7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:23:38.922977 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d78bd3d-1d20-4150-8fe1-6e88412a0d5f service cinder] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:23:38.923267 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d78bd3d-1d20-4150-8fe1-6e88412a0d5f service cinder] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:23:38.923566 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d78bd3d-1d20-4150-8fe1-6e88412a0d5f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:23:38.925547 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-8d78bd3d-1d20-4150-8fe1-6e88412a0d5f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:23:38.990401 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d78bd3d-1d20-4150-8fe1-6e88412a0d5f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:23:38.991912 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d78bd3d-1d20-4150-8fe1-6e88412a0d5f service cinder] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:23:39.003675 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1426/2852] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:23:38 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:23:40.361003 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc44b12-ded4-4a21-bde9-5422c7d956db None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:23:40.361503 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc44b12-ded4-4a21-bde9-5422c7d956db None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:23:40.361610 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc44b12-ded4-4a21-bde9-5422c7d956db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:23:40.381771 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-bdc44b12-ded4-4a21-bde9-5422c7d956db None None] MFA Rules not processed for user `4adc3e648c6d4bd9830dc064b9bc4e70`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:23:40.383074 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-bdc44b12-ded4-4a21-bde9-5422c7d956db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:23:40.390048 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1427/2853] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:23:40 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:23:40.620409 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d78bd3d-1d20-4150-8fe1-6e88412a0d5f service cinder] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:23:40.621928 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-8d78bd3d-1d20-4150-8fe1-6e88412a0d5f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:23:40.678625 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d78bd3d-1d20-4150-8fe1-6e88412a0d5f service cinder] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:23:40.682245 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5337436-dafd-4781-aec8-3df595314db6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51733781201243e8abf9eca39b207750', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:23:40.682880 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5337436-dafd-4781-aec8-3df595314db6 service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:23:40.683165 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5337436-dafd-4781-aec8-3df595314db6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:23:40.683444 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5337436-dafd-4781-aec8-3df595314db6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:23:40.685202 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-b5337436-dafd-4781-aec8-3df595314db6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:23:40.694429 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5337436-dafd-4781-aec8-3df595314db6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:23:40.695882 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5337436-dafd-4781-aec8-3df595314db6 service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:23:40.704595 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1427/2854] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:23:40 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:23:50.376804 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=76,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:23:50.700979 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=70 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:24:43.338233 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5ad6b2-64cc-4849-8276-9783c0298a89 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:24:43.338731 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5ad6b2-64cc-4849-8276-9783c0298a89 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:24:43.338833 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5ad6b2-64cc-4849-8276-9783c0298a89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:24:43.359119 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-3e5ad6b2-64cc-4849-8276-9783c0298a89 None None] MFA Rules not processed for user `4adc3e648c6d4bd9830dc064b9bc4e70`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:24:43.360367 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-3e5ad6b2-64cc-4849-8276-9783c0298a89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:24:43.367819 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1428/2855] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:24:43 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:24:43.403513 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5337436-dafd-4781-aec8-3df595314db6 service neutron] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:24:43.404942 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-b5337436-dafd-4781-aec8-3df595314db6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:24:43.459890 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5337436-dafd-4781-aec8-3df595314db6 service neutron] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:24:43.463381 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c778430-9f37-4a2d-b94f-b111d2a02587 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ea16528b1374bae9cf54603f7907bfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:24:43.464051 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c778430-9f37-4a2d-b94f-b111d2a02587 service glance] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:24:43.464337 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c778430-9f37-4a2d-b94f-b111d2a02587 service glance] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:24:43.464615 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c778430-9f37-4a2d-b94f-b111d2a02587 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:24:43.466355 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-7c778430-9f37-4a2d-b94f-b111d2a02587 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:24:43.539059 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c778430-9f37-4a2d-b94f-b111d2a02587 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:24:43.540587 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c778430-9f37-4a2d-b94f-b111d2a02587 service glance] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:24:43.552122 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1428/2856] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:24:43 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:24:44.299112 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5ad6b2-64cc-4849-8276-9783c0298a89 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:24:44.309605 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5ad6b2-64cc-4849-8276-9783c0298a89 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:24:44.313367 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1454b94-f213-422b-92f5-81b4b6298937 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51733781201243e8abf9eca39b207750', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:24:44.314012 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1454b94-f213-422b-92f5-81b4b6298937 service neutron] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:24:44.314303 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1454b94-f213-422b-92f5-81b4b6298937 service neutron] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:24:44.314588 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1454b94-f213-422b-92f5-81b4b6298937 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:24:44.316401 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-f1454b94-f213-422b-92f5-81b4b6298937 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:24:44.331145 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1454b94-f213-422b-92f5-81b4b6298937 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:24:44.332611 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1454b94-f213-422b-92f5-81b4b6298937 service neutron] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:24:44.344016 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1429/2857] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:24:44 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:24:53.546872 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=54 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:24:54.339079 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=9 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:25:05.150655 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1b44d4-4cdc-4109-a7fb-9a9f340e5cee None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:25:05.151136 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1b44d4-4cdc-4109-a7fb-9a9f340e5cee None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:25:05.151242 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1b44d4-4cdc-4109-a7fb-9a9f340e5cee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:25:05.170958 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-7a1b44d4-4cdc-4109-a7fb-9a9f340e5cee None None] MFA Rules not processed for user `3494ce75df62497bb737254fd1665a3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:25:05.172215 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-7a1b44d4-4cdc-4109-a7fb-9a9f340e5cee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:25:05.226502 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1429/2858] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:25:05 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:25:05.965817 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1454b94-f213-422b-92f5-81b4b6298937 service neutron] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:25:05.967292 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-f1454b94-f213-422b-92f5-81b4b6298937 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:25:06.036531 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1454b94-f213-422b-92f5-81b4b6298937 service neutron] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:25:06.041457 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f6d418-4e10-4e4d-8cd7-77e36ce17937 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8012afab6cac4358ab878732e58d0689', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:25:06.042101 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f6d418-4e10-4e4d-8cd7-77e36ce17937 service placement] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:25:06.042404 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f6d418-4e10-4e4d-8cd7-77e36ce17937 service placement] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:25:06.042676 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f6d418-4e10-4e4d-8cd7-77e36ce17937 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:25:06.044479 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-60f6d418-4e10-4e4d-8cd7-77e36ce17937 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:25:06.056230 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f6d418-4e10-4e4d-8cd7-77e36ce17937 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:25:06.057787 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f6d418-4e10-4e4d-8cd7-77e36ce17937 service placement] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:25:06.068280 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1430/2859] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:25:05 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:25:08.992995 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1b44d4-4cdc-4109-a7fb-9a9f340e5cee None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:25:08.994563 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-7a1b44d4-4cdc-4109-a7fb-9a9f340e5cee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:25:09.007440 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1b44d4-4cdc-4109-a7fb-9a9f340e5cee None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:25:09.011068 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efaea086-bd83-4119-952d-976a893af377 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51733781201243e8abf9eca39b207750', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:25:09.011763 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efaea086-bd83-4119-952d-976a893af377 service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:25:09.012053 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efaea086-bd83-4119-952d-976a893af377 service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:25:09.012336 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efaea086-bd83-4119-952d-976a893af377 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:25:09.014568 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-efaea086-bd83-4119-952d-976a893af377 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:25:09.025926 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efaea086-bd83-4119-952d-976a893af377 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:25:09.027492 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efaea086-bd83-4119-952d-976a893af377 service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:25:09.036725 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1430/2860] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:25:08 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:25:16.063412 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=32 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:25:19.032508 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=31 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:25:37.793923 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f6d418-4e10-4e4d-8cd7-77e36ce17937 service placement] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:25:37.795963 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-60f6d418-4e10-4e4d-8cd7-77e36ce17937 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:25:37.808788 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f6d418-4e10-4e4d-8cd7-77e36ce17937 service placement] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:25:37.813940 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b7df1f-af05-42d4-bbd8-aa8a71e4bbde service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8012afab6cac4358ab878732e58d0689', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:25:37.814588 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b7df1f-af05-42d4-bbd8-aa8a71e4bbde service placement] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:25:37.814874 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b7df1f-af05-42d4-bbd8-aa8a71e4bbde service placement] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:25:37.815180 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b7df1f-af05-42d4-bbd8-aa8a71e4bbde service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:25:37.816966 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-14b7df1f-af05-42d4-bbd8-aa8a71e4bbde service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:25:37.830215 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b7df1f-af05-42d4-bbd8-aa8a71e4bbde service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:25:37.831815 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b7df1f-af05-42d4-bbd8-aa8a71e4bbde service placement] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:25:37.843321 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1431/2861] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:25:37 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:25:37.859824 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efaea086-bd83-4119-952d-976a893af377 service neutron] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:25:37.861407 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-efaea086-bd83-4119-952d-976a893af377 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:25:37.931241 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efaea086-bd83-4119-952d-976a893af377 service neutron] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:25:37.935845 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28447861-672b-4030-a5c8-eaafde6ec06a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:25:37.936714 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28447861-672b-4030-a5c8-eaafde6ec06a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:25:37.937010 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28447861-672b-4030-a5c8-eaafde6ec06a admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:25:37.937295 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28447861-672b-4030-a5c8-eaafde6ec06a admin admin] PATH_INFO: `/v3/users/4adc3e648c6d4bd9830dc064b9bc4e70` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:25:37.938499 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28447861-672b-4030-a5c8-eaafde6ec06a admin admin] RBAC: Authorizing `identity:delete_user(user_id=4adc3e648c6d4bd9830dc064b9bc4e70)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:25:37.940459 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28447861-672b-4030-a5c8-eaafde6ec06a admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:25:37.984853 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-28447861-672b-4030-a5c8-eaafde6ec06a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4adc3e648c6d4bd9830dc064b9bc4e70'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:25:37.992283 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-28447861-672b-4030-a5c8-eaafde6ec06a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4adc3e648c6d4bd9830dc064b9bc4e70'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:25:38.000219 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-28447861-672b-4030-a5c8-eaafde6ec06a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4adc3e648c6d4bd9830dc064b9bc4e70'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:25:38.013351 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-28447861-672b-4030-a5c8-eaafde6ec06a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4adc3e648c6d4bd9830dc064b9bc4e70'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:25:38.013999 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-28447861-672b-4030-a5c8-eaafde6ec06a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4adc3e648c6d4bd9830dc064b9bc4e70'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:25:38.020293 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1431/2862] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Jul 25 12:25:37 2022] DELETE /identity/v3/users/4adc3e648c6d4bd9830dc064b9bc4e70 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:25:38.192922 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b7df1f-af05-42d4-bbd8-aa8a71e4bbde service placement] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:25:38.204082 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b7df1f-af05-42d4-bbd8-aa8a71e4bbde service placement] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:25:38.261424 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758cf56c-47d9-4984-b02c-5b1d61636f84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:25:38.262218 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758cf56c-47d9-4984-b02c-5b1d61636f84 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:25:38.262506 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758cf56c-47d9-4984-b02c-5b1d61636f84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:25:38.262785 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758cf56c-47d9-4984-b02c-5b1d61636f84 admin admin] PATH_INFO: `/v3/projects/a978671fc17b439eb73537f087a72e53` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:25:38.264481 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758cf56c-47d9-4984-b02c-5b1d61636f84 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a978671fc17b439eb73537f087a72e53)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:25:38.266373 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758cf56c-47d9-4984-b02c-5b1d61636f84 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:25:38.313482 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-758cf56c-47d9-4984-b02c-5b1d61636f84 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a978671fc17b439eb73537f087a72e53', 'request_id': 'req-758cf56c-47d9-4984-b02c-5b1d61636f84'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:25:38.320562 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-758cf56c-47d9-4984-b02c-5b1d61636f84 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a978671fc17b439eb73537f087a72e53', 'request_id': 'req-758cf56c-47d9-4984-b02c-5b1d61636f84'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:25:38.326041 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-758cf56c-47d9-4984-b02c-5b1d61636f84 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a978671fc17b439eb73537f087a72e53', 'request_id': 'req-758cf56c-47d9-4984-b02c-5b1d61636f84'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:25:38.336814 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-758cf56c-47d9-4984-b02c-5b1d61636f84 admin admin] The token cache is being invalidate because project a978671fc17b439eb73537f087a72e53 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80298) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 12:25:38.337099 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-758cf56c-47d9-4984-b02c-5b1d61636f84 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:25:38.337718 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-758cf56c-47d9-4984-b02c-5b1d61636f84 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:25:38.339465 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1432/2863] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Jul 25 12:25:38 2022] DELETE /identity/v3/projects/a978671fc17b439eb73537f087a72e53 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:25:38.344192 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9557f71-2f84-4aa2-80c5-3c81af0ce74b None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:25:38.344481 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9557f71-2f84-4aa2-80c5-3c81af0ce74b None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:25:38.344764 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9557f71-2f84-4aa2-80c5-3c81af0ce74b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:25:38.364585 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-f9557f71-2f84-4aa2-80c5-3c81af0ce74b None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:25:38.365778 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-f9557f71-2f84-4aa2-80c5-3c81af0ce74b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:25:38.422909 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1432/2864] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:25:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:25:38.431370 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75782a24-4506-4d8c-b8fd-1f6746b052d2 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:25:38.431726 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75782a24-4506-4d8c-b8fd-1f6746b052d2 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:25:38.432013 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75782a24-4506-4d8c-b8fd-1f6746b052d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:25:38.451255 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-75782a24-4506-4d8c-b8fd-1f6746b052d2 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:25:38.452508 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-75782a24-4506-4d8c-b8fd-1f6746b052d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:25:38.461814 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1433/2865] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:25:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:25:38.465921 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9557f71-2f84-4aa2-80c5-3c81af0ce74b None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:25:38.476096 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9557f71-2f84-4aa2-80c5-3c81af0ce74b None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:25:38.480632 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88545b72-3c09-44ee-b79b-a108b4898da8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:25:38.481289 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88545b72-3c09-44ee-b79b-a108b4898da8 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:25:38.481573 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88545b72-3c09-44ee-b79b-a108b4898da8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:25:38.481853 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88545b72-3c09-44ee-b79b-a108b4898da8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:25:38.482610 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88545b72-3c09-44ee-b79b-a108b4898da8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 12:25:38.482927 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88545b72-3c09-44ee-b79b-a108b4898da8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:25:38.484520 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88545b72-3c09-44ee-b79b-a108b4898da8 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:25:38.496596 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1433/2866] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Jul 25 12:25:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 12:25:38.500824 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75782a24-4506-4d8c-b8fd-1f6746b052d2 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:25:38.510092 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75782a24-4506-4d8c-b8fd-1f6746b052d2 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:25:38.515035 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac37ba0b-1cd6-49df-9be3-b1a29108bcf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:25:38.515706 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac37ba0b-1cd6-49df-9be3-b1a29108bcf7 admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:25:38.515872 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac37ba0b-1cd6-49df-9be3-b1a29108bcf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:25:38.516261 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac37ba0b-1cd6-49df-9be3-b1a29108bcf7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:25:38.516862 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac37ba0b-1cd6-49df-9be3-b1a29108bcf7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:25:38.519011 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac37ba0b-1cd6-49df-9be3-b1a29108bcf7 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:25:38.556151 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1434/2867] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Jul 25 12:25:38 2022] POST /identity/v3/projects => generated 373 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:25:38.560379 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88545b72-3c09-44ee-b79b-a108b4898da8 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:25:38.570473 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88545b72-3c09-44ee-b79b-a108b4898da8 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:25:38.574022 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:25:38.628291 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9534b71e-f7cd-49ba-acd3-81122cf84758 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:25:38.628959 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9534b71e-f7cd-49ba-acd3-81122cf84758 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:25:38.629247 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9534b71e-f7cd-49ba-acd3-81122cf84758 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:25:38.629525 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9534b71e-f7cd-49ba-acd3-81122cf84758 admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:25:38.630240 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9534b71e-f7cd-49ba-acd3-81122cf84758 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:25:38.632212 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9534b71e-f7cd-49ba-acd3-81122cf84758 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:25:38.653265 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1434/2868] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Jul 25 12:25:38 2022] POST /identity/v3/users => generated 287 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 12:25:38.657801 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac37ba0b-1cd6-49df-9be3-b1a29108bcf7 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:25:38.668690 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac37ba0b-1cd6-49df-9be3-b1a29108bcf7 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:25:38.673485 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28efd22c-9c26-4454-a787-cef04a36aa27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:25:38.674141 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28efd22c-9c26-4454-a787-cef04a36aa27 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:25:38.674435 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28efd22c-9c26-4454-a787-cef04a36aa27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:25:38.674718 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28efd22c-9c26-4454-a787-cef04a36aa27 admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:25:38.675452 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28efd22c-9c26-4454-a787-cef04a36aa27 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:25:38.677082 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28efd22c-9c26-4454-a787-cef04a36aa27 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:25:38.686968 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1435/2869] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Jul 25 12:25:38 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:25:38.690772 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9534b71e-f7cd-49ba-acd3-81122cf84758 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:25:38.700794 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9534b71e-f7cd-49ba-acd3-81122cf84758 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:25:38.705545 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b2f5fa-b050-414f-86c2-d5c363297404 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:25:38.706189 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b2f5fa-b050-414f-86c2-d5c363297404 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:25:38.706475 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b2f5fa-b050-414f-86c2-d5c363297404 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:25:38.706755 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b2f5fa-b050-414f-86c2-d5c363297404 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:25:38.707476 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b2f5fa-b050-414f-86c2-d5c363297404 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:25:38.709083 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b2f5fa-b050-414f-86c2-d5c363297404 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:25:38.720895 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1435/2870] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Jul 25 12:25:38 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:25:38.724641 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28efd22c-9c26-4454-a787-cef04a36aa27 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:25:38.735229 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28efd22c-9c26-4454-a787-cef04a36aa27 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:25:38.739914 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370f8c72-94d2-400e-b05f-56331cf6cbf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:25:38.740796 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370f8c72-94d2-400e-b05f-56331cf6cbf4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:25:38.741094 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370f8c72-94d2-400e-b05f-56331cf6cbf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:25:38.741381 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370f8c72-94d2-400e-b05f-56331cf6cbf4 admin admin] PATH_INFO: `/v3/projects/376f79202b2d46b4bff1b67d8b10add6/users/edb6dd7dc1af4bc79386df797dab6e1e/roles/472313ae69174578824df0bed6390744` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:25:38.756010 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370f8c72-94d2-400e-b05f-56331cf6cbf4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=376f79202b2d46b4bff1b67d8b10add6, user_id=edb6dd7dc1af4bc79386df797dab6e1e, role_id=472313ae69174578824df0bed6390744)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:25:38.758662 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370f8c72-94d2-400e-b05f-56331cf6cbf4 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:25:38.773141 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1436/2871] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Jul 25 12:25:38 2022] PUT /identity/v3/projects/376f79202b2d46b4bff1b67d8b10add6/users/edb6dd7dc1af4bc79386df797dab6e1e/roles/472313ae69174578824df0bed6390744 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:25:38.778221 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb6632a-e624-4ee2-a443-8668434d26d3 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:25:38.778549 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb6632a-e624-4ee2-a443-8668434d26d3 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:25:38.778828 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb6632a-e624-4ee2-a443-8668434d26d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:25:38.797910 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-bbb6632a-e624-4ee2-a443-8668434d26d3 None None] MFA Rules not processed for user `0fad2691baf0472d99015f9f123f8d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:25:38.799115 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-bbb6632a-e624-4ee2-a443-8668434d26d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:25:38.856158 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1436/2872] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:25:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:25:38.865854 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370f8c72-94d2-400e-b05f-56331cf6cbf4 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:25:38.867692 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-370f8c72-94d2-400e-b05f-56331cf6cbf4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:25:38.874269 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:25:38.925559 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370f8c72-94d2-400e-b05f-56331cf6cbf4 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:25:38.929250 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65dc07b0-2d8e-4f11-b365-dae5f4c73312 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51733781201243e8abf9eca39b207750', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:25:38.929889 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dc07b0-2d8e-4f11-b365-dae5f4c73312 service neutron] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:25:38.930177 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dc07b0-2d8e-4f11-b365-dae5f4c73312 service neutron] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:25:38.930459 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dc07b0-2d8e-4f11-b365-dae5f4c73312 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:25:38.939368 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65dc07b0-2d8e-4f11-b365-dae5f4c73312 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:25:38.940802 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65dc07b0-2d8e-4f11-b365-dae5f4c73312 service neutron] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:25:38.950139 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1437/2873] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:25:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:25:42.832038 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da10bbea-2ade-4c10-a04a-b2fa1465431e None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:25:42.832605 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da10bbea-2ade-4c10-a04a-b2fa1465431e None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:25:42.832605 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da10bbea-2ade-4c10-a04a-b2fa1465431e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:25:42.864172 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-da10bbea-2ade-4c10-a04a-b2fa1465431e None None] MFA Rules not processed for user `edb6dd7dc1af4bc79386df797dab6e1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:25:42.865576 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-da10bbea-2ade-4c10-a04a-b2fa1465431e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:25:42.936340 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1437/2874] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:25:42 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:25:42.940844 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65dc07b0-2d8e-4f11-b365-dae5f4c73312 service neutron] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:25:42.951594 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65dc07b0-2d8e-4f11-b365-dae5f4c73312 service neutron] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:25:42.956322 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2404b6-8754-462d-b0d0-1607217bd99d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:25:42.956989 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2404b6-8754-462d-b0d0-1607217bd99d admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:25:42.957285 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2404b6-8754-462d-b0d0-1607217bd99d admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:25:42.957806 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2404b6-8754-462d-b0d0-1607217bd99d admin admin] PATH_INFO: `/v3/projects` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:25:42.958550 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2404b6-8754-462d-b0d0-1607217bd99d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:25:42.961266 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2404b6-8754-462d-b0d0-1607217bd99d admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:25:42.992965 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1438/2875] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Jul 25 12:25:42 2022] POST /identity/v3/projects => generated 373 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 12:25:42.994040 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da10bbea-2ade-4c10-a04a-b2fa1465431e None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:25:43.006115 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da10bbea-2ade-4c10-a04a-b2fa1465431e None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:25:43.019055 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:25:43.077138 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88917793-13dc-450d-83bb-6fb3209adcf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:25:43.077825 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88917793-13dc-450d-83bb-6fb3209adcf7 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:25:43.078119 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88917793-13dc-450d-83bb-6fb3209adcf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:25:43.078407 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88917793-13dc-450d-83bb-6fb3209adcf7 admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:25:43.079141 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88917793-13dc-450d-83bb-6fb3209adcf7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:25:43.081143 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88917793-13dc-450d-83bb-6fb3209adcf7 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:25:43.106640 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1438/2876] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Jul 25 12:25:42 2022] POST /identity/v3/users => generated 293 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 12:25:43.107635 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2404b6-8754-462d-b0d0-1607217bd99d admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:25:43.119085 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2404b6-8754-462d-b0d0-1607217bd99d admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:25:43.123873 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beea86ca-61ed-412d-a9b7-ea3eefc3aebe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:25:43.124552 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beea86ca-61ed-412d-a9b7-ea3eefc3aebe admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:25:43.124845 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beea86ca-61ed-412d-a9b7-ea3eefc3aebe admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:25:43.125248 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beea86ca-61ed-412d-a9b7-ea3eefc3aebe admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:25:43.125996 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beea86ca-61ed-412d-a9b7-ea3eefc3aebe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:25:43.127672 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beea86ca-61ed-412d-a9b7-ea3eefc3aebe admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:25:43.138820 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1439/2877] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Jul 25 12:25:43 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 12:25:43.142967 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88917793-13dc-450d-83bb-6fb3209adcf7 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:25:43.154514 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88917793-13dc-450d-83bb-6fb3209adcf7 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:25:43.159234 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1d10c5-2bee-44af-97a3-5a1713d06a35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:25:43.160197 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1d10c5-2bee-44af-97a3-5a1713d06a35 admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:25:43.160484 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1d10c5-2bee-44af-97a3-5a1713d06a35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:25:43.160756 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1d10c5-2bee-44af-97a3-5a1713d06a35 admin admin] PATH_INFO: `/v3/projects/eea4ecec7feb427fbbb769f7246047a3/users/66d62b13d5b14bc7afa08748a6a5f1df/roles/a7d20885b6d94928a34b3cc5b3a00006` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:25:43.177103 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1d10c5-2bee-44af-97a3-5a1713d06a35 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eea4ecec7feb427fbbb769f7246047a3, user_id=66d62b13d5b14bc7afa08748a6a5f1df, role_id=a7d20885b6d94928a34b3cc5b3a00006)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:25:43.179920 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1d10c5-2bee-44af-97a3-5a1713d06a35 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:25:43.195024 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1439/2878] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Jul 25 12:25:43 2022] PUT /identity/v3/projects/eea4ecec7feb427fbbb769f7246047a3/users/66d62b13d5b14bc7afa08748a6a5f1df/roles/a7d20885b6d94928a34b3cc5b3a00006 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:25:43.198973 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beea86ca-61ed-412d-a9b7-ea3eefc3aebe admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:25:43.209512 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beea86ca-61ed-412d-a9b7-ea3eefc3aebe admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:25:43.265266 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8a9bec-ea33-49bf-8b70-02d9077e79b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:25:43.265931 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8a9bec-ea33-49bf-8b70-02d9077e79b7 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:25:43.266231 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8a9bec-ea33-49bf-8b70-02d9077e79b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:25:43.266519 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8a9bec-ea33-49bf-8b70-02d9077e79b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:25:43.267250 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8a9bec-ea33-49bf-8b70-02d9077e79b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:25:43.269147 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8a9bec-ea33-49bf-8b70-02d9077e79b7 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:25:43.280211 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1440/2879] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Jul 25 12:25:43 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 12:25:43.284379 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1d10c5-2bee-44af-97a3-5a1713d06a35 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:25:43.295239 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1d10c5-2bee-44af-97a3-5a1713d06a35 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:25:43.299815 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cafb13-6468-46cd-9f1b-793a33c6ec8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:25:43.300682 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cafb13-6468-46cd-9f1b-793a33c6ec8e admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:25:43.300967 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cafb13-6468-46cd-9f1b-793a33c6ec8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:25:43.301248 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cafb13-6468-46cd-9f1b-793a33c6ec8e admin admin] PATH_INFO: `/v3/projects/eea4ecec7feb427fbbb769f7246047a3/users/66d62b13d5b14bc7afa08748a6a5f1df/roles/a7d20885b6d94928a34b3cc5b3a00006` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:25:43.303227 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19cafb13-6468-46cd-9f1b-793a33c6ec8e admin admin] RBAC: Authorizing `identity:create_grant(project_id=eea4ecec7feb427fbbb769f7246047a3, user_id=66d62b13d5b14bc7afa08748a6a5f1df, role_id=a7d20885b6d94928a34b3cc5b3a00006)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:25:43.305898 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19cafb13-6468-46cd-9f1b-793a33c6ec8e admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:25:43.316032 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1440/2880] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Jul 25 12:25:43 2022] PUT /identity/v3/projects/eea4ecec7feb427fbbb769f7246047a3/users/66d62b13d5b14bc7afa08748a6a5f1df/roles/a7d20885b6d94928a34b3cc5b3a00006 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:25:47.817028 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c3e631-d675-4e5c-9f10-889719b20547 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:25:47.818149 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c3e631-d675-4e5c-9f10-889719b20547 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:25:47.818645 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c3e631-d675-4e5c-9f10-889719b20547 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:25:47.865375 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-12c3e631-d675-4e5c-9f10-889719b20547 None None] MFA Rules not processed for user `66d62b13d5b14bc7afa08748a6a5f1df`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:25:47.866918 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-12c3e631-d675-4e5c-9f10-889719b20547 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:25:47.954586 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:25:47.970128 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1441/2881] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:25:47 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:25:47.986686 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cafb13-6468-46cd-9f1b-793a33c6ec8e admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:25:47.989200 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-19cafb13-6468-46cd-9f1b-793a33c6ec8e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:25:48.085818 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cafb13-6468-46cd-9f1b-793a33c6ec8e admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:25:48.091713 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d6d9ca-2593-4dce-8d29-ff5989ae3340 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f67142a1193d47c7bd33d7040268233c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:25:48.092386 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d6d9ca-2593-4dce-8d29-ff5989ae3340 service nova] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:25:48.092675 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d6d9ca-2593-4dce-8d29-ff5989ae3340 service nova] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:25:48.092955 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d6d9ca-2593-4dce-8d29-ff5989ae3340 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:25:48.153315 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d6d9ca-2593-4dce-8d29-ff5989ae3340 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:25:48.154909 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d6d9ca-2593-4dce-8d29-ff5989ae3340 service nova] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:25:48.163685 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1441/2882] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:25:47 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:25:49.136902 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebc9f64-819e-4330-b95e-f630d9e4175b None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:25:49.137423 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebc9f64-819e-4330-b95e-f630d9e4175b None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:25:49.137513 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebc9f64-819e-4330-b95e-f630d9e4175b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:25:49.157047 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-4ebc9f64-819e-4330-b95e-f630d9e4175b None None] MFA Rules not processed for user `edb6dd7dc1af4bc79386df797dab6e1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:25:49.158269 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-4ebc9f64-819e-4330-b95e-f630d9e4175b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:25:49.165853 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1442/2883] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:25:49 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 12:25:58.159756 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=93,keystone:INSERT=5 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:25:59.151724 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=5 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:07.353300 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d6d9ca-2593-4dce-8d29-ff5989ae3340 service nova] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:07.355173 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-01d6d9ca-2593-4dce-8d29-ff5989ae3340 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:26:07.381698 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d6d9ca-2593-4dce-8d29-ff5989ae3340 service nova] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:07.388178 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ead410-c8b6-43c4-801f-aa4ce141a2d9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f67142a1193d47c7bd33d7040268233c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:07.388843 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ead410-c8b6-43c4-801f-aa4ce141a2d9 service nova] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:07.389132 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ead410-c8b6-43c4-801f-aa4ce141a2d9 service nova] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:07.389687 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ead410-c8b6-43c4-801f-aa4ce141a2d9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:07.391734 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-c2ead410-c8b6-43c4-801f-aa4ce141a2d9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:26:07.405754 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ead410-c8b6-43c4-801f-aa4ce141a2d9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:07.407463 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ead410-c8b6-43c4-801f-aa4ce141a2d9 service nova] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:07.421070 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1442/2884] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:26:07 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:26:17.178900 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebc9f64-819e-4330-b95e-f630d9e4175b None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:17.193921 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebc9f64-819e-4330-b95e-f630d9e4175b None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:17.199362 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cba921-454b-49ab-be63-6b270cce28c1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f67142a1193d47c7bd33d7040268233c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:17.200044 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cba921-454b-49ab-be63-6b270cce28c1 service nova] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:17.200334 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cba921-454b-49ab-be63-6b270cce28c1 service nova] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:17.200614 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cba921-454b-49ab-be63-6b270cce28c1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:17.202463 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-e9cba921-454b-49ab-be63-6b270cce28c1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:26:17.216071 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9cba921-454b-49ab-be63-6b270cce28c1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:17.217722 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9cba921-454b-49ab-be63-6b270cce28c1 service nova] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:17.228507 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1443/2885] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:26:17 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 12:26:17.414284 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=6 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:19.424495 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ead410-c8b6-43c4-801f-aa4ce141a2d9 service nova] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:26:19.425906 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-c2ead410-c8b6-43c4-801f-aa4ce141a2d9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:26:19.480816 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ead410-c8b6-43c4-801f-aa4ce141a2d9 service nova] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:26:19.484382 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0666ad5f-45b8-4a77-82c7-13d44e4541e4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51733781201243e8abf9eca39b207750', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:26:19.485013 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0666ad5f-45b8-4a77-82c7-13d44e4541e4 service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:26:19.485301 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0666ad5f-45b8-4a77-82c7-13d44e4541e4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:26:19.485580 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0666ad5f-45b8-4a77-82c7-13d44e4541e4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:26:19.494275 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0666ad5f-45b8-4a77-82c7-13d44e4541e4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:26:19.495725 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0666ad5f-45b8-4a77-82c7-13d44e4541e4 service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:26:19.505731 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1443/2886] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:26:19 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:26:27.224036 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=6 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:26:29.500852 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=26 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:04.559889 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cba921-454b-49ab-be63-6b270cce28c1 service nova] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:04.561297 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-e9cba921-454b-49ab-be63-6b270cce28c1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:27:04.619485 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cba921-454b-49ab-be63-6b270cce28c1 service nova] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:04.623153 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6152eec4-43cd-4611-815c-5ad537519e1c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f89ce6f0cf04373a414d927fa0f8a7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:04.623828 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6152eec4-43cd-4611-815c-5ad537519e1c service cinder] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:04.624123 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6152eec4-43cd-4611-815c-5ad537519e1c service cinder] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:04.624408 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6152eec4-43cd-4611-815c-5ad537519e1c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:04.626158 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-6152eec4-43cd-4611-815c-5ad537519e1c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:27:04.684958 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6152eec4-43cd-4611-815c-5ad537519e1c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:04.686425 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6152eec4-43cd-4611-815c-5ad537519e1c service cinder] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:04.696081 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1444/2887] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:27:04 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:27:11.525064 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a5922b-7db0-4551-a9c6-8a31cbe612ed None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:11.526096 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a5922b-7db0-4551-a9c6-8a31cbe612ed None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:11.526096 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a5922b-7db0-4551-a9c6-8a31cbe612ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:11.545493 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-09a5922b-7db0-4551-a9c6-8a31cbe612ed None None] MFA Rules not processed for user `3494ce75df62497bb737254fd1665a3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 12:27:11.546717 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-09a5922b-7db0-4551-a9c6-8a31cbe612ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:27:11.553569 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1444/2888] 10.222.0.42 () {58 vars in 958 bytes} [Mon Jul 25 12:27:11 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 12:27:14.691320 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=48 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:21.539671 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=3 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:27:53.095884 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6152eec4-43cd-4611-815c-5ad537519e1c service cinder] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:27:53.097291 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-6152eec4-43cd-4611-815c-5ad537519e1c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:27:53.166170 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6152eec4-43cd-4611-815c-5ad537519e1c service cinder] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:27:53.171166 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c3b379-8a46-4a38-8450-a12af47f91f8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8012afab6cac4358ab878732e58d0689', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:27:53.171929 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c3b379-8a46-4a38-8450-a12af47f91f8 service placement] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:27:53.172223 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c3b379-8a46-4a38-8450-a12af47f91f8 service placement] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:27:53.172508 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c3b379-8a46-4a38-8450-a12af47f91f8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:27:53.174334 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-99c3b379-8a46-4a38-8450-a12af47f91f8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:27:53.185913 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c3b379-8a46-4a38-8450-a12af47f91f8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:27:53.188105 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c3b379-8a46-4a38-8450-a12af47f91f8 service placement] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:27:53.198026 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1445/2889] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:27:53 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:28:03.193754 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=32 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:28:29.983063 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a5922b-7db0-4551-a9c6-8a31cbe612ed None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:28:29.993058 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a5922b-7db0-4551-a9c6-8a31cbe612ed None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:28:29.996639 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0449a64c-6d40-4237-baec-c6b1d493add7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51733781201243e8abf9eca39b207750', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:28:29.997273 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0449a64c-6d40-4237-baec-c6b1d493add7 service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:28:29.997560 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0449a64c-6d40-4237-baec-c6b1d493add7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:28:29.997839 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0449a64c-6d40-4237-baec-c6b1d493add7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:28:29.999691 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-0449a64c-6d40-4237-baec-c6b1d493add7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:28:30.010630 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0449a64c-6d40-4237-baec-c6b1d493add7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:28:30.012106 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0449a64c-6d40-4237-baec-c6b1d493add7 service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:28:30.021252 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1445/2890] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:28:29 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 12:28:30.112759 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c3b379-8a46-4a38-8450-a12af47f91f8 service placement] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:28:30.122491 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c3b379-8a46-4a38-8450-a12af47f91f8 service placement] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:28:30.126146 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471d82ad-66e8-49a6-bfe7-5424268f9765 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51733781201243e8abf9eca39b207750', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:28:30.126788 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471d82ad-66e8-49a6-bfe7-5424268f9765 service neutron] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:28:30.127604 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471d82ad-66e8-49a6-bfe7-5424268f9765 service neutron] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:28:30.127890 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471d82ad-66e8-49a6-bfe7-5424268f9765 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:28:30.129704 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-471d82ad-66e8-49a6-bfe7-5424268f9765 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:28:30.139747 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471d82ad-66e8-49a6-bfe7-5424268f9765 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:28:30.141198 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471d82ad-66e8-49a6-bfe7-5424268f9765 service neutron] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:28:30.150519 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1446/2891] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:28:30 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:28:30.225469 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0449a64c-6d40-4237-baec-c6b1d493add7 service neutron] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:28:30.234268 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0449a64c-6d40-4237-baec-c6b1d493add7 service neutron] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:28:30.238912 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08014f0-7ccb-456f-a976-5c2eeda623d9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51733781201243e8abf9eca39b207750', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:28:30.239593 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08014f0-7ccb-456f-a976-5c2eeda623d9 service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:28:30.239887 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08014f0-7ccb-456f-a976-5c2eeda623d9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:28:30.240169 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08014f0-7ccb-456f-a976-5c2eeda623d9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:28:30.241957 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-a08014f0-7ccb-456f-a976-5c2eeda623d9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:28:30.257804 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08014f0-7ccb-456f-a976-5c2eeda623d9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:28:30.259284 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08014f0-7ccb-456f-a976-5c2eeda623d9 service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:28:30.270678 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1446/2892] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:28:30 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 12:28:40.146631 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=6 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:28:40.265510 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=12 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:28:59.141234 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471d82ad-66e8-49a6-bfe7-5424268f9765 service neutron] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:28:59.156940 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471d82ad-66e8-49a6-bfe7-5424268f9765 service neutron] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:28:59.163454 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34c838a-12ce-4c50-aa7d-af0c5f76779e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f67142a1193d47c7bd33d7040268233c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:28:59.164221 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34c838a-12ce-4c50-aa7d-af0c5f76779e service nova] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:28:59.164586 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34c838a-12ce-4c50-aa7d-af0c5f76779e service nova] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:28:59.164954 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34c838a-12ce-4c50-aa7d-af0c5f76779e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:28:59.167107 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-f34c838a-12ce-4c50-aa7d-af0c5f76779e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:28:59.181973 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34c838a-12ce-4c50-aa7d-af0c5f76779e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:28:59.183620 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34c838a-12ce-4c50-aa7d-af0c5f76779e service nova] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:28:59.195675 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1447/2893] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:28:59 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:29:09.190420 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=6 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:29:11.608307 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08014f0-7ccb-456f-a976-5c2eeda623d9 service neutron] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:29:11.616979 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08014f0-7ccb-456f-a976-5c2eeda623d9 service neutron] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:29:11.620706 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e7b360-77dd-464c-bec5-dea934b8887f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51733781201243e8abf9eca39b207750', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:29:11.621287 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e7b360-77dd-464c-bec5-dea934b8887f service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:29:11.621577 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e7b360-77dd-464c-bec5-dea934b8887f service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:29:11.621856 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e7b360-77dd-464c-bec5-dea934b8887f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:29:11.623625 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-c7e7b360-77dd-464c-bec5-dea934b8887f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:29:11.688163 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e7b360-77dd-464c-bec5-dea934b8887f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:29:11.689601 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e7b360-77dd-464c-bec5-dea934b8887f service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:29:11.699622 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1447/2894] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:29:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:29:17.620396 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34c838a-12ce-4c50-aa7d-af0c5f76779e service nova] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:29:17.624994 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-f34c838a-12ce-4c50-aa7d-af0c5f76779e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:29:17.640421 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34c838a-12ce-4c50-aa7d-af0c5f76779e service nova] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:29:17.645707 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f409120-bc30-4a43-adf3-2f0d8bf42354 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:29:17.646677 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f409120-bc30-4a43-adf3-2f0d8bf42354 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:29:17.647039 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f409120-bc30-4a43-adf3-2f0d8bf42354 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:29:17.647376 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f409120-bc30-4a43-adf3-2f0d8bf42354 admin admin] PATH_INFO: `/v3/users/3494ce75df62497bb737254fd1665a3b` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:29:17.648892 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f409120-bc30-4a43-adf3-2f0d8bf42354 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3494ce75df62497bb737254fd1665a3b)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:29:17.651366 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f409120-bc30-4a43-adf3-2f0d8bf42354 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:29:17.698907 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-0f409120-bc30-4a43-adf3-2f0d8bf42354 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3494ce75df62497bb737254fd1665a3b'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:29:17.699496 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-0f409120-bc30-4a43-adf3-2f0d8bf42354 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3494ce75df62497bb737254fd1665a3b'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:29:17.707670 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-0f409120-bc30-4a43-adf3-2f0d8bf42354 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3494ce75df62497bb737254fd1665a3b'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:29:17.721502 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-0f409120-bc30-4a43-adf3-2f0d8bf42354 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3494ce75df62497bb737254fd1665a3b'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:29:17.729463 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-0f409120-bc30-4a43-adf3-2f0d8bf42354 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3494ce75df62497bb737254fd1665a3b'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:29:17.735634 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1448/2895] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Jul 25 12:29:17 2022] DELETE /identity/v3/users/3494ce75df62497bb737254fd1665a3b => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:29:17.915562 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e7b360-77dd-464c-bec5-dea934b8887f service neutron] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:29:17.925682 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e7b360-77dd-464c-bec5-dea934b8887f service neutron] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:29:17.982726 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9663d5b8-2e22-43e1-a2c2-75a78e98d5b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:29:17.983725 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9663d5b8-2e22-43e1-a2c2-75a78e98d5b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:29:17.984021 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9663d5b8-2e22-43e1-a2c2-75a78e98d5b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:29:17.984303 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9663d5b8-2e22-43e1-a2c2-75a78e98d5b8 admin admin] PATH_INFO: `/v3/projects/141003bbc94b41569683aef4c15b46f1` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:29:17.985495 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9663d5b8-2e22-43e1-a2c2-75a78e98d5b8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=141003bbc94b41569683aef4c15b46f1)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:29:17.987416 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9663d5b8-2e22-43e1-a2c2-75a78e98d5b8 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:29:18.033549 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-9663d5b8-2e22-43e1-a2c2-75a78e98d5b8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '141003bbc94b41569683aef4c15b46f1', 'request_id': 'req-9663d5b8-2e22-43e1-a2c2-75a78e98d5b8'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:29:18.040630 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-9663d5b8-2e22-43e1-a2c2-75a78e98d5b8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '141003bbc94b41569683aef4c15b46f1', 'request_id': 'req-9663d5b8-2e22-43e1-a2c2-75a78e98d5b8'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:29:18.046463 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-9663d5b8-2e22-43e1-a2c2-75a78e98d5b8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '141003bbc94b41569683aef4c15b46f1', 'request_id': 'req-9663d5b8-2e22-43e1-a2c2-75a78e98d5b8'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:29:18.057254 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-9663d5b8-2e22-43e1-a2c2-75a78e98d5b8 admin admin] The token cache is being invalidate because project 141003bbc94b41569683aef4c15b46f1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 12:29:18.057542 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-9663d5b8-2e22-43e1-a2c2-75a78e98d5b8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:29:18.058170 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-9663d5b8-2e22-43e1-a2c2-75a78e98d5b8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:29:18.059736 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1448/2896] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Jul 25 12:29:17 2022] DELETE /identity/v3/projects/141003bbc94b41569683aef4c15b46f1 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:29:18.063703 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f409120-bc30-4a43-adf3-2f0d8bf42354 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:29:18.065135 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-0f409120-bc30-4a43-adf3-2f0d8bf42354 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:29:18.079166 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f409120-bc30-4a43-adf3-2f0d8bf42354 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:29:18.136137 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a773d6-44f2-48a5-aff3-09e03435fa49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:29:18.137048 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a773d6-44f2-48a5-aff3-09e03435fa49 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:29:18.137336 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a773d6-44f2-48a5-aff3-09e03435fa49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:29:18.137615 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a773d6-44f2-48a5-aff3-09e03435fa49 admin admin] PATH_INFO: `/v3/users/28e5eae4e13945768c0c440cfbf7d429` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:29:18.138829 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a773d6-44f2-48a5-aff3-09e03435fa49 admin admin] RBAC: Authorizing `identity:delete_user(user_id=28e5eae4e13945768c0c440cfbf7d429)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:29:18.140784 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a773d6-44f2-48a5-aff3-09e03435fa49 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:29:18.186729 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-c0a773d6-44f2-48a5-aff3-09e03435fa49 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '28e5eae4e13945768c0c440cfbf7d429'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:29:18.187320 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-c0a773d6-44f2-48a5-aff3-09e03435fa49 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '28e5eae4e13945768c0c440cfbf7d429'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:29:18.194884 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-c0a773d6-44f2-48a5-aff3-09e03435fa49 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '28e5eae4e13945768c0c440cfbf7d429'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:29:18.207459 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-c0a773d6-44f2-48a5-aff3-09e03435fa49 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '28e5eae4e13945768c0c440cfbf7d429'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:29:18.209479 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:29:18.215745 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-c0a773d6-44f2-48a5-aff3-09e03435fa49 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '28e5eae4e13945768c0c440cfbf7d429'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:29:18.223046 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1449/2897] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Jul 25 12:29:18 2022] DELETE /identity/v3/users/28e5eae4e13945768c0c440cfbf7d429 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:29:18.391599 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9663d5b8-2e22-43e1-a2c2-75a78e98d5b8 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:29:18.393091 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-9663d5b8-2e22-43e1-a2c2-75a78e98d5b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:29:18.406062 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9663d5b8-2e22-43e1-a2c2-75a78e98d5b8 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:29:18.427012 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:29:18.464735 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32209f82-f64d-4717-b676-d73a2e6c2528 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:29:18.465510 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32209f82-f64d-4717-b676-d73a2e6c2528 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:29:18.465791 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32209f82-f64d-4717-b676-d73a2e6c2528 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:29:18.466068 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32209f82-f64d-4717-b676-d73a2e6c2528 admin admin] PATH_INFO: `/v3/projects/5922b8528101418b84ca9b5b53f49cc5` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:29:18.467232 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32209f82-f64d-4717-b676-d73a2e6c2528 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5922b8528101418b84ca9b5b53f49cc5)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:29:18.469175 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32209f82-f64d-4717-b676-d73a2e6c2528 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:29:18.514598 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-32209f82-f64d-4717-b676-d73a2e6c2528 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5922b8528101418b84ca9b5b53f49cc5', 'request_id': 'req-32209f82-f64d-4717-b676-d73a2e6c2528'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:29:18.522020 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-32209f82-f64d-4717-b676-d73a2e6c2528 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5922b8528101418b84ca9b5b53f49cc5', 'request_id': 'req-32209f82-f64d-4717-b676-d73a2e6c2528'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:29:18.527847 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-32209f82-f64d-4717-b676-d73a2e6c2528 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5922b8528101418b84ca9b5b53f49cc5', 'request_id': 'req-32209f82-f64d-4717-b676-d73a2e6c2528'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:29:18.542814 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-32209f82-f64d-4717-b676-d73a2e6c2528 admin admin] The token cache is being invalidate because project 5922b8528101418b84ca9b5b53f49cc5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 12:29:18.543106 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-32209f82-f64d-4717-b676-d73a2e6c2528 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:29:18.543782 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-32209f82-f64d-4717-b676-d73a2e6c2528 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:29:18.545346 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1449/2898] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Jul 25 12:29:18 2022] DELETE /identity/v3/projects/5922b8528101418b84ca9b5b53f49cc5 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:29:28.219985 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=4 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:29:28.536996 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=40,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:29:55.594823 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a773d6-44f2-48a5-aff3-09e03435fa49 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:29:55.596538 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-c0a773d6-44f2-48a5-aff3-09e03435fa49 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:29:55.655637 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a773d6-44f2-48a5-aff3-09e03435fa49 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:29:55.721597 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a986f52-a468-4d4a-b368-e698f39a8853 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51733781201243e8abf9eca39b207750', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:29:55.722252 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a986f52-a468-4d4a-b368-e698f39a8853 service neutron] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:29:55.722544 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a986f52-a468-4d4a-b368-e698f39a8853 service neutron] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:29:55.722832 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a986f52-a468-4d4a-b368-e698f39a8853 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:29:55.724676 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-3a986f52-a468-4d4a-b368-e698f39a8853 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:29:55.802147 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a986f52-a468-4d4a-b368-e698f39a8853 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:29:55.803629 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a986f52-a468-4d4a-b368-e698f39a8853 service neutron] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:29:55.815241 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1450/2899] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:29:55 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:29:56.992101 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32209f82-f64d-4717-b676-d73a2e6c2528 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:29:56.993532 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-32209f82-f64d-4717-b676-d73a2e6c2528 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:29:57.006177 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32209f82-f64d-4717-b676-d73a2e6c2528 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:29:57.009721 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51d1cae-47f8-4c35-b943-558f1226c1b7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51733781201243e8abf9eca39b207750', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:29:57.010370 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51d1cae-47f8-4c35-b943-558f1226c1b7 service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:29:57.010654 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51d1cae-47f8-4c35-b943-558f1226c1b7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:29:57.010932 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51d1cae-47f8-4c35-b943-558f1226c1b7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:29:57.012648 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-e51d1cae-47f8-4c35-b943-558f1226c1b7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:29:57.025601 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51d1cae-47f8-4c35-b943-558f1226c1b7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:29:57.027021 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51d1cae-47f8-4c35-b943-558f1226c1b7 service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:29:57.038076 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1450/2900] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:29:56 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:30:05.743481 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a986f52-a468-4d4a-b368-e698f39a8853 service neutron] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:30:05.745496 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-3a986f52-a468-4d4a-b368-e698f39a8853 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:30:05.805255 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:30:05.810170 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a986f52-a468-4d4a-b368-e698f39a8853 service neutron] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:30:05.815069 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d575f7-9ef7-499d-aac9-6eec1a57b8f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:30:05.815962 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d575f7-9ef7-499d-aac9-6eec1a57b8f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:30:05.816248 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d575f7-9ef7-499d-aac9-6eec1a57b8f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:30:05.816527 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d575f7-9ef7-499d-aac9-6eec1a57b8f5 admin admin] PATH_INFO: `/v3/users/edb6dd7dc1af4bc79386df797dab6e1e` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:30:05.817700 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d575f7-9ef7-499d-aac9-6eec1a57b8f5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=edb6dd7dc1af4bc79386df797dab6e1e)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:30:05.819633 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d575f7-9ef7-499d-aac9-6eec1a57b8f5 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:30:05.865137 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-e9d575f7-9ef7-499d-aac9-6eec1a57b8f5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'edb6dd7dc1af4bc79386df797dab6e1e'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:30:05.865729 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-e9d575f7-9ef7-499d-aac9-6eec1a57b8f5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'edb6dd7dc1af4bc79386df797dab6e1e'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:30:05.874425 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-e9d575f7-9ef7-499d-aac9-6eec1a57b8f5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'edb6dd7dc1af4bc79386df797dab6e1e'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:30:05.886469 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-e9d575f7-9ef7-499d-aac9-6eec1a57b8f5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'edb6dd7dc1af4bc79386df797dab6e1e'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:30:05.893387 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-e9d575f7-9ef7-499d-aac9-6eec1a57b8f5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'edb6dd7dc1af4bc79386df797dab6e1e'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:30:05.899156 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1451/2901] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Jul 25 12:30:05 2022] DELETE /identity/v3/users/edb6dd7dc1af4bc79386df797dab6e1e => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:30:06.082160 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51d1cae-47f8-4c35-b943-558f1226c1b7 service neutron] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:30:06.094068 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51d1cae-47f8-4c35-b943-558f1226c1b7 service neutron] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:30:06.152996 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f06204-c4d9-4a82-8691-d755c2b2c9dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:30:06.153780 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f06204-c4d9-4a82-8691-d755c2b2c9dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:30:06.154070 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f06204-c4d9-4a82-8691-d755c2b2c9dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:30:06.154353 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f06204-c4d9-4a82-8691-d755c2b2c9dd admin admin] PATH_INFO: `/v3/projects/376f79202b2d46b4bff1b67d8b10add6` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:30:06.155520 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f06204-c4d9-4a82-8691-d755c2b2c9dd admin admin] RBAC: Authorizing `identity:delete_project(project_id=376f79202b2d46b4bff1b67d8b10add6)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:30:06.157442 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f06204-c4d9-4a82-8691-d755c2b2c9dd admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:30:06.203444 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-44f06204-c4d9-4a82-8691-d755c2b2c9dd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '376f79202b2d46b4bff1b67d8b10add6', 'request_id': 'req-44f06204-c4d9-4a82-8691-d755c2b2c9dd'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:30:06.210077 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-44f06204-c4d9-4a82-8691-d755c2b2c9dd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '376f79202b2d46b4bff1b67d8b10add6', 'request_id': 'req-44f06204-c4d9-4a82-8691-d755c2b2c9dd'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:30:06.216639 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-44f06204-c4d9-4a82-8691-d755c2b2c9dd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '376f79202b2d46b4bff1b67d8b10add6', 'request_id': 'req-44f06204-c4d9-4a82-8691-d755c2b2c9dd'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:30:06.227273 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-44f06204-c4d9-4a82-8691-d755c2b2c9dd admin admin] The token cache is being invalidate because project 376f79202b2d46b4bff1b67d8b10add6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 12:30:06.227584 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-44f06204-c4d9-4a82-8691-d755c2b2c9dd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:30:06.228213 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-44f06204-c4d9-4a82-8691-d755c2b2c9dd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:30:06.230261 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1451/2902] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Jul 25 12:30:06 2022] DELETE /identity/v3/projects/376f79202b2d46b4bff1b67d8b10add6 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:30:06.234353 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d575f7-9ef7-499d-aac9-6eec1a57b8f5 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:30:06.235801 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-e9d575f7-9ef7-499d-aac9-6eec1a57b8f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:30:06.248608 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d575f7-9ef7-499d-aac9-6eec1a57b8f5 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:30:06.303446 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e210a195-61f9-4b6e-b8fd-b57d722f1e92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:30:06.304332 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e210a195-61f9-4b6e-b8fd-b57d722f1e92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:30:06.304616 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e210a195-61f9-4b6e-b8fd-b57d722f1e92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:30:06.304891 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e210a195-61f9-4b6e-b8fd-b57d722f1e92 admin admin] PATH_INFO: `/v3/users/66d62b13d5b14bc7afa08748a6a5f1df` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:30:06.306107 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e210a195-61f9-4b6e-b8fd-b57d722f1e92 admin admin] RBAC: Authorizing `identity:delete_user(user_id=66d62b13d5b14bc7afa08748a6a5f1df)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:30:06.308063 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e210a195-61f9-4b6e-b8fd-b57d722f1e92 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:30:06.358585 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-e210a195-61f9-4b6e-b8fd-b57d722f1e92 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '66d62b13d5b14bc7afa08748a6a5f1df'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:30:06.359284 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-e210a195-61f9-4b6e-b8fd-b57d722f1e92 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '66d62b13d5b14bc7afa08748a6a5f1df'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:30:06.366632 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-e210a195-61f9-4b6e-b8fd-b57d722f1e92 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '66d62b13d5b14bc7afa08748a6a5f1df'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:30:06.378687 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-e210a195-61f9-4b6e-b8fd-b57d722f1e92 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '66d62b13d5b14bc7afa08748a6a5f1df'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:30:06.383931 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:30:06.386473 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-e210a195-61f9-4b6e-b8fd-b57d722f1e92 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '66d62b13d5b14bc7afa08748a6a5f1df'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:30:06.394001 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1452/2903] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Jul 25 12:30:06 2022] DELETE /identity/v3/users/66d62b13d5b14bc7afa08748a6a5f1df => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:30:06.563287 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f06204-c4d9-4a82-8691-d755c2b2c9dd admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:30:06.564970 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-44f06204-c4d9-4a82-8691-d755c2b2c9dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:30:06.578443 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f06204-c4d9-4a82-8691-d755c2b2c9dd admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:30:06.639165 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba90e7b-ec16-4d8b-87ef-7a6250804b0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fad2691baf0472d99015f9f123f8d32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '827585a7cfe8411987f9b81102c1c103', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:30:06.640055 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba90e7b-ec16-4d8b-87ef-7a6250804b0e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:30:06.640336 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba90e7b-ec16-4d8b-87ef-7a6250804b0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:30:06.640609 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba90e7b-ec16-4d8b-87ef-7a6250804b0e admin admin] PATH_INFO: `/v3/projects/eea4ecec7feb427fbbb769f7246047a3` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:30:06.641778 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba90e7b-ec16-4d8b-87ef-7a6250804b0e admin admin] RBAC: Authorizing `identity:delete_project(project_id=eea4ecec7feb427fbbb769f7246047a3)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:30:06.643701 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba90e7b-ec16-4d8b-87ef-7a6250804b0e admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:30:06.664204 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:30:06.692055 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-8ba90e7b-ec16-4d8b-87ef-7a6250804b0e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eea4ecec7feb427fbbb769f7246047a3', 'request_id': 'req-8ba90e7b-ec16-4d8b-87ef-7a6250804b0e'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:30:06.703361 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-8ba90e7b-ec16-4d8b-87ef-7a6250804b0e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eea4ecec7feb427fbbb769f7246047a3', 'request_id': 'req-8ba90e7b-ec16-4d8b-87ef-7a6250804b0e'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:30:06.709375 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-8ba90e7b-ec16-4d8b-87ef-7a6250804b0e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eea4ecec7feb427fbbb769f7246047a3', 'request_id': 'req-8ba90e7b-ec16-4d8b-87ef-7a6250804b0e'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:30:06.721523 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-8ba90e7b-ec16-4d8b-87ef-7a6250804b0e admin admin] The token cache is being invalidate because project eea4ecec7feb427fbbb769f7246047a3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 12:30:06.721810 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-8ba90e7b-ec16-4d8b-87ef-7a6250804b0e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:30:06.722501 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-8ba90e7b-ec16-4d8b-87ef-7a6250804b0e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 12:30:06.724088 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1452/2904] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Jul 25 12:30:06 2022] DELETE /identity/v3/projects/eea4ecec7feb427fbbb769f7246047a3 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 12:30:16.393094 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=2 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:30:16.718445 dv-cinder-797608-61-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=16,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 12:32:57.489601 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e210a195-61f9-4b6e-b8fd-b57d722f1e92 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 12:32:57.491039 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-e210a195-61f9-4b6e-b8fd-b57d722f1e92 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:32:57.629624 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e210a195-61f9-4b6e-b8fd-b57d722f1e92 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 12:32:57.687646 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd8294a-009d-4d44-99ad-119269fdd8c2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8012afab6cac4358ab878732e58d0689', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9f62b8f37304b8a9a4bb54205347577', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 12:32:57.688298 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd8294a-009d-4d44-99ad-119269fdd8c2 service placement] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 12:32:57.689080 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd8294a-009d-4d44-99ad-119269fdd8c2 service placement] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 12:32:57.689386 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd8294a-009d-4d44-99ad-119269fdd8c2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 12:32:57.691176 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-dfd8294a-009d-4d44-99ad-119269fdd8c2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 12:32:57.702599 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd8294a-009d-4d44-99ad-119269fdd8c2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 12:32:57.704241 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd8294a-009d-4d44-99ad-119269fdd8c2 service placement] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 12:32:57.714800 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1453/2905] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Jul 25 12:32:57 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 12:33:07.710484 dv-cinder-797608-61-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=89 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}